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ABSTRACT 

This monograph describes three types of evaluation 
that are potentially useful to school-based clinics: needo 
assessments, process evaluations, and impact evaluations. Two 
important methodological principles are involved: (3) collecting 
multiple kinds of data with multiple methods; and (2) collecting 
comparison data. Student needs can be identified by using student and 
parent surveys or community data. Various management information 
systems are useful for collecting process data on clinic utilization. 
Student satisfaction can also be easily measured, and site visits by 
other professionals can help identify clinic strengths and 
weaknesses. Outcomes are harder to measure because clinics typically 
serve relatively small student populations for any particular health 
outcome and the students who patronize the clinic differ from those 
who do not. Diffusion and self-selection are major problems. The 
traditional solution is to use an experimental design (based on a 
large sample) that measures the clinic's impact upon participants. 
The best method of measuring tne clinic's impact on the entire 
student body is to collect survey data from all the students or merge 
lists of student n-jnes with birth records or other public records. 
Rates can then be compared over time or between clinic and similar 
comparison schools. Appendices contain the student health survey, an 
explanation of the Center for Population Options School-Based Clinic 
Management Information System, and a description of methods for 
assessing a clinic's pregnancy prevention program. (13 references) 
(MLH) 
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Preface 



Evaluation is a necessary and important part of the development of all programs, including 
school-based clinics. Well-designed evaluation can help clinic administrators and others understand 
more clearly what needs their potential users may have, what their clinic programs are currently 
doing, what impact the programs have upon participants, and how ihe programs can be improved. 
Of course, evaluation may also be necessary to demonstrate success and thereby maintain or secure 
program funding. 

School-based clinics around the country are in different stages of development and have 
different research or evaluation needs. The purpose of this monograph is to help those who are 
contemplating or actually designing an evaluation project This monograph discusses three types of 
evaluation, their purposes, the variables to be measured, the study designs, the target populations, 
and the sources of data. 

Many professionals considering an evaluation of clinic programs are clinic administrators or 
practitioners who have relatively little research background; thus, this monograph covers some basic 
research principles that app'y to clinic evaluation. Other professionals designing an evaluation are 
expenenced researchers; thty may want to skim parts of this monograph quickly and focus on those 
parts that are most useful to them, for example, the sections on experimental design and generating 
birth rate data. Throughout ths text, readings are suggested for those who seek more information 
on a particular topic. 

Much of the information included in this document is based on lessons learned from a large 
evaluation of school-based clinics conducted by the Center for Population Options and from the 
collective wisdom of advisory board members who have had extensive experience with school-based 
clinics, with evaluations of pregnancy prevention programs, and with research more generally. 
Some of the methodological innovations of this monograph (e.g., tho methods of measuring school 
birth rates more accurately) were developed explicitly for this project Other materials developed or 
refined for this project include the Student Health Survey and an inventory of important 
characteristics of clinic reproductive health programs. Finally, available in a separate volume is a 
management information system that was also developed for this project and that can provide much 
of the process and outcome dr'a discussed below. 

Because this review of evaluation methods grew out of a project focusing upon reproductive 
health, many of the examples and materials developed are related to adolescent sexual activity, 
contraceptive use, and pregnancy. However, most methods described in the monograph can and 
should be used more generally to measure other process and outcome goals of clinics. 

In sum, clinic educators, practitioners, administrators, and researchers should all find this 
monograph useful when designing a research and evaluation plan for their clinics and for 
identifying refined and tested methods to conduct their research activities. 
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I. Introduction 



The Increasing Importance of School-Based Clinics and Their Evaluation 

Health clinics located in schools or on school campuses represent an increasingly important 
health delivery system. Many health professionals view school-based clinics as a promising 
approach to improving adolescent health care and to reducing important adolescent risk-taking 
behaviors, particularly those leading to teenage pregnancy, substance abuse, violence, and 
depression. Consequendy, there has been a rapid growth in these clinics-about every two years the 
number of clinics has doubled-and by the beginning of 1990 there were more than 150 clinics in 
junior and senior high schools throughout the country. In addition there are many more clinics that 
are school-linked clinics which serve students in schools, but are physically located across the 
street, or nearby. 

Evaluation of school-based clinics is necessary because program planners need accurate 
information on the health needs of the students, the ability of tneir clinics to provide needed health 
services, the impact of these services, and the wrys to improve those services. Evaluation is also 
necessary for funding agencies who need evidence that their money is well spent, and by policy- 
makers and planners who need to know whether specific programs meet the needs of students and 
have the desired outcomes. 

Although local and nationwide evaluations of school-based clinics have been and are being 
conducted, many questions remain unanswered. There is not a single student need nor a single 
clinic model; rather student needs differ from one community to another, and clinic personne' 
services, and procedures also differ. In addition, it is difficult to evaluate clinics, and past and 
current evaluations, like any evaluation, have had their limitations. Thus, additional evaluations of 
specific clinics are needed. 

There is an additional reason to evaluate clinics. The actual process of participating in an 
evaluation is commonly helpful; it facilitates the more careful delineation of program goals and 
more realistic expectations for program success. 



Types of Evaluation 



There are three types of evaluation that are potentially useful to school-based clinics: needs 
assessments, process evaluations, and impact evaluations. These are summarized in Figure 1. 

Needs assessments. Typically, needs assessments collect data to answer questions about 
what student needs are not being met, what health problems, health behaviors and risk-taking 
behaviors should be addressed, whether a school-based clinic is needed, which services are most 
critical, and how members of the school or the community feel about having a school-based clinics 
Typical questions include: What percentage of students are severely depressed? What percentage 
engage in unprotected sexual behavior? What percentage use drugs? What percentage do not 
receive needed medical care? Is it difficult for parents/guardians to take them to needed care' 
What services do the parents believe the clinic should provide? 

Because these are questions that arise during the design of a clinic, needs assessments 
should logically be conducted either before a clinic opens or during the first year of operation 
However, even after a clinic has been open for several years, it may be useful to conduct a new 
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needs assessment to determine whether student needs have changed and whether the clinic or other 
community services are meeting ti;nse needs. 

Process evaluations. Process evaluations assess the extent to which clinic services are 
actually being provided to students. They monitor the success of day-to-day program operations 
Piey address such questions as: How many students are enrolled in the clinic? How many 
actually use the clinic? Which services do they typically receive? Does utilization differ bv gender 
o- poverty status .' What kinds of health and psychosocial problems are diagnosed' Do the clinics 
ha^« high appointment compliance rates? Does the clinic provide high quality care? Does it 
provide health care in a cost effective manner? 

This type of evaluation is necessarily conducted after the clinic has begun operation. 
However, process evaluations can be more complete and valid if they are designed during the 
planning stages of the clinic. Otherwise process data that should be collected during the first year 
of operation might be lost, or alternatively needed statistics may have to be subsequently culled 
from clinic records in a time-consuming manner. 

Impact evaluations. Outcome or impact evaluations are conducted to determine the 
effectiveness or clinic programs in producing specified desired outcomes. For example they 
address questions such as: Have the students received more health care than they would otherwise 
have received? Has the duration of time since they received needed dental care decreased? Did 
the clinic programs reduce depression or other psycho-social problems? Did they improve 
adolescents' eating or exercise habits? Did they reduce unprotected sexual activity or substance 
abuse? Did they improve absenteeism and dropout rates? 

Impact evaluations can only be conducted after clinics have opened. Normally, they should 
not be conducted until programs specifically designed to address particular goals have been well 
developed and implemented and have served many students. Often this takes three or more years 
of clime operatioa However, it is important to collect baseline data before a clinic opens, and thus 
impact evaluations are more useful when designed before a clinic opens. 

Whereas virtually all clinics should employ some methods of assessing student needs and 
measuring clinic services, clinics should approach impact evaluations far more cautiously. On the 
one hand, impact evaluations can answer critical questions about the ability of programs to produce 
desired changes. On the other hand, some impact evaluations may require methodological expertise 
and substantial funding and can be difficult and time consuming. They may require the cooperation 
of many people, such as the clinic staff, the school administration, and the students in the school, 
and the results may be disappointing, either because they are inconclusive or because they indicate 
the clintc is not as successful as believed. Of course, not all impact evaluations require major 
*..urts or resources, but clinics deciding to participate in impact evaluations should be aware of 
these considerations before making a commitment to participate. 

Needs assessments, process evaluations, and impact evaluations are not mutually exclusive; 
indeed, they may be reinforcing. For example, student survey data collected for a needs assessment 
may be used ?s baseline data in an impact evaluation. Similariy, the number of students served in 
a particular health area may help determine what type of experimental design to use in an impact 
evaluation, and certainly, process data can help explain why clinic programs do or do not have 
desired impacts. 

Once the type of evaluation is determined, then decisions need to be made about the 
research questions to be asked, the variables to be measured, the study design, the target 
populations, and the sources of data to be collected. Possible alternatives are summarized in 
Figure 1. 
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The organization of this monograph ap ,roximates that framework. Fim, this monograph 
discusses principles and considerations that apply to all three types of evaluation. Then there are 
sections on needs assessments, process evaluations, and impact evaluations. Topics such as 
sampling, outcome measures, school surveys, and birth rate data which apply primarily to impact 
evaluations and to a lesser extent to needs assessments are discussed following the section on 
impact evaluation. 



Figure 1 

TYPE OF NEEDS PROCESS IMPACT 

EVALUATION ASSESSMENTS EVALUATION EVALUATION 



PURPOSE 



VARIABLES 
TO BE 
MEASURED 



To assist program 
development by 
identifying and 
demonstrating 
student needs 

Needs of 
students 



To monitor 
and assess 
program 
services 



Clinic utilization, 
compliance, 
user satisfaction 



To evaluate 
the impact 
and effectiveness 
of programs 



Improvement in 
healthful behavior, 
decrease in risk-taking 
behaviors 



STUDY 
DESIGN 



Cross-sectional 



Cross-sectional, 
time-series 



Experimental, 
quasi-experimental 



TARGET 
POPULATION 



Students, 
pcrents, 

school personnel 



Clinic users 



Clinic users, 
all students 



SOURCE 
OF DATA 



Student surveys, 
parent surveys, 
community 
statistics 



Clinic records, 
user surveys, 
student surveys, 
parent surveys 



Clinic records, 
student surveys, 
school records, 
public records 
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II. Methodological Issues Relevant to All 
School-Based Clinic Evaluation 



Evaluating Health Clinics in a School Setting 

Although it is common to evaluate educational programs in a school environment, it is far 
less common to evaluate health clinics in a school environment. Doing so provides special 
opportunities for research that do not occur when health clinics are evaluated in the broader 
community. In particular, most school-based clinics have a reasonably small, well-defined, and 
specific target population-namely, the students in the schools. All the members of the target 
population are identified by name, and thus these names can be checked against other lists of 
names, and rates or other statistics can be determined. And to make matters still more conducive 
to research, nearly all the members of the target population of school-based clinics are regularly 
brought together in one convenient location-namely the school, where information can be collected 
from them through surveys or other means. In addition, other institutions, such as the school itself 
already collect useful aggregate information on the students (e.g., absenteeism and dropout rates). 
These characteristics of school settings can facilitate needs assessments, process evaluations, and 
impact evaluations. 

On the other hand, the school setting may also make it more difficult to coUect certain 
kinds of data (e.g., data on sexual activity or drug use), may require that additional types of 
consent be obtained, and may make it more difficult to employ certain types of study designs such 
as expenmental designs. These issues are more fully discussed below. 

Two Important Methodological Principles 

CoK rting multiple kinds of data with multiple methods . When conducting research on 
school-based tames, it is important to collect and analyze more than one type of data Every 
m ? h0 J ° f coUectin g data has its own flaws and sources of error. Data collection methods that are 
quite different are likely to have different sources of error. Consequently, if an evaluator collects 
two or more types of data and they support conflicting conclusions, then the evaluator knows that 
one or more of the data collection methods contains misleading errors. On the other hand, if two 
or more different types of data support the same conclusion, then the evaluator can have greater 
confidence in that conclusion than if that conclusion were based upon only one type of data. 

This general principle has several implications. It suggests that when conducting needs 
assessments, it m*y be useful to estimate student needs both with student surveys and with 
community statistics on health behaviors. It suggests that surveying both students and their parnts 
is useful. When conducting process evaluations, data should ideally be collected from clinic 
records, from student or parent surveys, and from interviews with students. When conducting 
impact evaluations, outcome data should be integrated with process data and different types of 
outcome data should be collected. 

An example involving substance abuse or teenage pregnancy may help illustrate this For 
the needs assessment, a school-wide survey can provide estimates of the number (or percentage) of 
students who use drugs or have sex. These data can be compared with community reports on 
adolescent substance abuse (possibly from the police department) and with community statistics on 



teenage pregnancy or birth rates (perhaps from the health department). For the process evaluation, 
clinic records can provide data on the number of students participating in any type of substance 
abuse or reproductive health program. These can be compared with data from school surveys on 
clinic utilization. By comparing the data from the needs assessments showing how many students 
need to participate in substance abuse or reproductive health programs with data from the process 
evaluanon showing how many students actually participated, it is possible to determine the extent to 
which the clinic is meeting the students' needs in these areas. If many of the students in need are 
being served, then conducting an impact evaluation may be appropriate. For the impact evaluation, 
a second survey of students can be conducted and it can be examined to determine whether the 
number of students using drugs or having unprotected sex has decreased. These results can also be 
compared with public records to determine if the number of students arrested for using drugs has 
decreased or if the school birth rate has declined. Thus, by collecting and integrating different 
kinds of data, a fuller, more accurate picture of the clinic can be created. 

However, evaluators should not allow their use of multiple methods and multiple types of 
data to lull them into complacency about the quality of the data being collected. CoUecting several 
kinds of data poorly is not necessarily better than collecting one kind of data rigorously This is 
especially true if several types of poor data collection methods are insufficiendy sensitive to d 
a clinic s actual impact and would therefore lead to the erroneous conclusion that the clinic had no 
impact. 

Collecting comparison data. Much evaluation is based upon comparisons. For example, 
in a needs assessment, data showing that 30% of the students had not seen a medical doctor for 
two years or that 12% got pregnant the previous year become more meaningful statistics when they 
are compared with statistics for local, state, or national populations. In a process evaluation 
knowing that 50% of the students are enrolled and that the clinic had U00 encounters becomes 
more meaningful when these data are compared with figures for other school clinics or other health 
clinics. Finally, in an impact evaluation, comparison data are an especially critical component of 
any type of experimental design; without such data, conclusions cannot be reached about the impact 
of clinics. Knowing that 30% of the sexually active males used condoms the last time they had 
sex is not meaningful unless it is also known whether this represented an increase or decrease and 
whether this increase was greater than the national or community-wide increase. 

Important Steps in an Evaluation 

The steps below are commonly completed during a needs assessment, a process evaluation 
or an impact evaluation (adapted from Briidis, Korenbrot and Brown, 1986): 

1. Identify important evaluation questions; select important needs to be measured or 
important goals and objectives to be evaluated. 

2. Identify potential data resources and constraints. 

3. Select the study design. 

4. Select the methods of collecting data. 

5. Select a sampling plan. 

6. Develop and test data collection methods. 

7. Collect data. 

8. Analyze data. 

9. Report findings. 

The order of these sieps is not invariant; sometimes it is more useful to complete some of the 
design steps simultaneously or in a different order. 
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Getting Started-Some First Steps 

There are a number of steps that can facilitate the implementation of an evaluation. 

Gct hc| P - Depending upon both the evaluation skills that the evaluators already have and 
the desired scope of the evaluation, there are different types of help that can be obtained. Anyone 
planning to do research on clinics should contact those who are already doing research on clinics or 
on similar programs. From these researchers, it is possible to obtain sample clinic encounter forms, 
needs assessments, other survey questionnaires, summaries of possible evaluation methods, and 
summaries of previous research. And, of course, it is possible to learn from their successes and 
mistakes. 

There are also relevant conferences that one can attend. These include conferences with 
sessions on school-based clinics (e.g., those sponsored by the Center for Population Options or by 
the American Public Health AssocLiion), those on adolescence (e.g., the Society for Research on 
Adolescence), and those on evaluation more generally (e.g., the American Evaluation Society). All 
of them discuss relevant evaluation issues. 

If there is limited funding, ihen a nearby university may be contacted where faculty 
members or graduate students may be willing to lend expertise and play either a minor or major 
*oi* in the evaluation. 

Finally, there are helpful materials to read on evaluation. Two particularly helpful resources 
are Evaluatio n of Pregnancy Prevention Programs in the School Context by Laurie Zabin and 
Marilyn Hirsch and Evaluating Programs Aimed at Preventing Teenage Pregnancies by Josefina 
Card. 

Plan the evaluation and program simultaneously . As suggested above, if possible, plan 
the needs assessment, the methods for collecting process data, and the design of the impact 
evaluation (if there will be one) at the same time that you plan the clinic itself. Too oiUn, 
professionals implementing clinics focus primarily upon the innumerable difficulties in opening a 
clinic and therefore pay little attention to evaluation. Unfortunately, this shortsightedness may have 
negative consequences for the impact evaluation, because once the clinic is open, it may no longer 
be possible to collect baseline data unaffected by the clinic's presence, to use certain experimental 
designs, or to collect important clinic data. 

Recognize the social context of evaluating school-based clinics . Although there are many 
valid reasons for evaluating school-based clinics, it should also be recognized that school-based 
clinics are scrutinized more than many other health programs. This is partly because they represent 
a relatively new form of health care delivery and partly because many school-based health clinics 
address adolescent sexuality and openuc in a highly politicized environment 

Furthermore, the reporting of some previous results may have produced unrealistic 
expectations for the impact of clinics upon adolescent risk-taking behaviors. After all, most 
research demonstrates that it is difficult to change adolescent risk-taking behavior, especially sexual 
behavior. Furthermore, school-based clinics attempt to address these difficult-to-change behaviors 
with a broad range of services addressing a broad range of needs. And they commonly do so in 
communities where many services may already exist for adolescents and others. Thus, it seems 
unreasonable to expect that additional services provided through school-based clinics will cause 
substantial decreases in behaviors such as skipping school, cigarette smoking, substance abuse and 
unprotected sexual intercourse. 
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This has implications for the methods selected to evaluate climes; it also has implications 
for the expectauons that people should have for the results of the evaluation. Neither clinic 
personnel nor the larger community should expect to find dramatic changes in behavior. If there 
are unreasonable expectations, then those expectations should be made more realistic before the 
evaluation is undertaken and its results reported. 



III. Conducting Needs Assessments 

Although this monograph focuses primarily upon evaluating the impact of school-based 
clinic programs, needs assessments are useful for any planned or newly opened program These 
assessments can assist program managers to determine more accurately the health services that 
should be provided or emphasized in the clinic. These data can also provide to the broader 
community or to potential hinders documentation of the need to provide specific services. 
Furthermore, as has been noted above, some types of needs assessment data can also be used to 
assess whether clinics are meeting student needs and having an impact upon them. 

Data on the health needs of students can be collected from a variety of different sources- 
students, their parents, school personnel, other health providers in the clinic, and public records. 

Probably the single best method of collecting data about student health needs is to conduct 
a survey of all (or a representative sample of) the students in the schooL The survey questionnaire 
should include questions on a variety of health areas, for example, existing access to health 
services, existence of health insurance, use of health services when needed, health problems or 
concerns, rental health problems such as depression or stress, health behaviors such as nutrition 
and exercise, and risk-taking behaviors such as unprotected sex or substance abuse. They should 
also include sufficient background or demographic information to identify which groups of students 
are most in need (eg., information about gender, age, ethnicity, or poverty status). An example of 
such a questionnaire is included in the Appendix A. Methods of administering this needs 
assessment questionnaire to the students are discussed below in the section on "Sources and Tvpes 
of Dau for Need; Assessments and Impact Evaluations." 

Although parents have less information about student needs in some areas (e.g , risk-taking 
behaviors), they have more knowledge of other types of information (e.g., availability of health 
insurance and ease of taking their adolescent to health care providers). Thus, a survey of all 
parents (or a representative sample of parents) may usefully supplement the student survey. If 
parent surveys are to be reasonably representative, they need to be mailed directly to parents or 
guardians rather than sent home with the students. A cover letter from the principal emphasizing 
the the importance of the survey and a self-addressed, stamped return envelope should be enclosed. 

These survey data can be supplemented with school and community data. These often have 
the advantage that they have already been collected and may be relatively easy to obtain School 
personnel can normally provide statistics on absenteeism and dropout rates. Sometimes, they can 
provide either insights or hard data on the prevalence of fights on campus, drug use on campus 
pregnancies, and other health-related behav'ors. 

Finally, vario>is community agencies can often provide data on the use (and non-use) of 
community health agencies by adolescents, on pregnancy or birth rates and on other health 
problems of youth. For a fuller discussion of these statistics and how to use them to justify the 
need for a clinic and to design a clinic, see School-Based Health Clinics: A Guide to Implementing 
Programs by Hadley, Lovick, and Kirby. 6 
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IV. Conducting a Process Evaluation 



The primary goal of school-based clinics is to provide health care to young people Thus 
it is very important to conduct process evaluations of school-based clinics to determine how many 
students use the clinics, how often they use the clinic, for what health purposes, etc. Beyond these 
basic utilization questions, process evaluation can be done to determine appointment compliance 
student satisfaction, cost effectiveness and effectiveness of clinic operations. These kinds of 
questions can be answered with a process evaluation which focuses upon the participation in a 
program rather than the needs of the students or the impact of the program on them. 

Process data are also important as a first step in an outcome evaluation. They can 
determine how many students use a particular service or participate in a particular program. If only 
a few people participate, then the program may not have a measurable impact, at least not upon the 
entire school, rhus, the process evaluation can detennine whether it is worthwhile to conduct an 
outcome evaluation, and if so, how. 



Clinic Utilization 

Most clinics can and do provide evidence for clinic utilization by summarizing the clinic's 
visits and encounters. They do this by keeping some written or computerized record of each visit 
or encounter with each patient. These records can be tallied by hand or preferably by a computer 
and the resulting frequencies reported. 

At a minimum, visit or encounter forms should include the patient's ID number (to 
distinguish between duplicated and unduplicated visits) and information about each encounter (eg 

^°l e J f w J siU typc of staff P 6130 " *«>■ Additional information is also useful. (See the 
CPO SCMIS System for an example.) 

When counting visits or encounters, it is important to distinguish between the number of 
duplicated and unduplicated visits, so that different questions can be answered: How many different 
students used the clinic? How many visits did students make to the clinic? On the average how 
many times did each student use the clinic? 

By including the purpose of the visit on the encounter form, the number of visits for 
different purposes can be determined. The same is true for diagnoses. Some diagnoses, such as 
sexual abuse or severe medical problems, can be very important, even if there are only small 
numbers of them. Others may be important for identifying students to participate in an outcome 
evaluation. 

Commonly it is also useful to provide breakdowns by age, year in school, sex, existence of 
family insurance, type of visit, and other characteristics of the students or the visit Then these 
frequencies can answer obvious, but important, questions such as: How many males (and' females) 
visited the clinic for sports physicals? How many females (or males) used the clinic for family 
planning? How many freshmen used the clinic? How many seniors? How many were without 
other sources of medical care or without any form of insurance? 

In order to produce these breakdowns, this information about age, gender, year in school, 
etc., must either be provided on each encounter form, or alternatively be provided on a registration 
form which is linked by identification number to each visit or encounter. The lattei option 
necessitates the use of a computerized system. 
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By simply dividing the numbers of students seen in the clinic by the appropriate number of 
students m the school, it is also possible to answer questions such as: What percentage of the 
males (or females) use the clinic? What percentage of females (or males) use it for family 
planning? 1 

Most schools can provide data on the number of students enrolled. However, in many 
schools, the enrollment figures (which are used in the denominators in some of the percentages 
above) decline substantially during the year and the number of students officially enrolled may 
significantly exceed the number of students who actually attend school. Thus, selecting the correct 
enrollment number may be difficult. Zabin and Hirsch (1988) provide a good discussion of this 
problem. 

The number of students receiving specific services can also be compared with the number 
of students needing that service as determined by a school survey or needs assessment. For 
example, if a school has 1,000 female students, if the needs assessment indicates that 50% of them 
are currently sexually active, and if the clinic provides services to 300 of them, then it can be 
estimated that the clinic is providing tamily planning services to 300 out of 500 or 60% of the 
sexually active females. Similarly, estimates can be provided for the proportion of students using 
drugs who are given drug counseling, the proportion of students without recent physical exams who 
are given such exams, or the proportion of students with specific health concerns who attend the 
clinic for those health concerns. 

An important goal of some clinics is to reach those students who .re especially high risk 
for health and other problems. The methods described above can assess the extent to which the 
clinic is reaching students engaging in specific risk-taking behaviors. 

By collecting these kinds of data each year, it is easy to produce trend data and to 
determine whether these numbers or percentages are increasing as rapidly as expected over time. 

Although counts of patients, visits or encounters and percentages based upon them may 
appear simple and easy to collect, agreement dees need to be reached on several definitions. First, 
it is important to define rather precisely what constitutes an encounter. Otherwise, ambiguity can ' 
arise in determining exactly how much interaction or what kinds of interaction must take place in 
order to count an interaction as an encounter. For example, if a student discusses her diet for 15 
minutes with a nutritionist, this interaction would clearly count as an encounter. On the other hand, 
if that same girl tells the nutritionist in passing that her diet is going well, that interaction would 
not constitute an encounter. Clea«iy there is pressure to count minimal interactions as encounters, 
because doing so increases the apparent clinic activity. However, doing so is misleading. An 
appropriate rule of thumb is that if an encounter is sufficiently important to write something in that 
student's health record, then that encounter should be counted; otherwise it should not. 

A similar problem arises when a student sees more than one person during the same c ,: Mc 
visit In general, it is wise to foUow the BCHS regulations which stipulate that only those sta^ 
persons who make independent decisions regarding a patient should be counted on the encounter 
form. For example, if a student sees a nurse-practitioner about family planning and a nutritionist 
about her diet and they made independent decisions about use of birth control puis and her diet 
this would count as one visit and two encounters. On the other hand, if the same student first 
visited die nurse-practitioner for fr jUy planning and then the health educator for more information 
on family planning, this would count as both one visit and one encounter. 

The CPO S chool-Based Clinic Management Information System . The Center for 
Population Options has developed a compr ;hensive SBC management information system. It is 
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available free of charge from CPO. This system answers the kinds of questions identified above 
plus many more. It includes both data collection forms and software for processing and analyzing 
encounter data. 

Only three forms (a registration form, standard visit form, and case closing form) are 
required to use the system These three fonns can produce about 50 tables of data. In addition, 
six optional fonns allow the collection of additional data and the creation of additional tables. 

The software is designed to operate on IBM compatible micro-computers and is written in 
dBase Ill-Plus, a popular softwsr* iwiguage that can be revised by sites if needed. The software is 
a menu-driven system that is easy for people without micro-computer experience to learn and use. 
The tables produce statistics on the major purposes of visits, diagnoses made, types of medical 
procedures and counseling services provided, types of external referrals made, staff utilization 
patterns, and a wide variety of other useful summary data. Appendix B contains a fuller 
description of the SCMIS system and a list of the questions that the system can answer. 

Though clinic records are most often used to evaluate clinic utilization, other methods can 
be used as well. School-wide surveys can provide estimates of the percentage of the school 
enrollment served and the reasons for clinic visits. Focus groups can provide more qualitative 
infonnation on students' perceptions of clinic operations. 



Appointment Compliance Rates 

Appointment compliance rates are important because high appointment compliance may be 
critical to providing proper care, and partly because school-based clinics may have especially high 
compliance rates. After all, the location of these clinics does make it is easy for the students to go 
to the clinic and for the clinic to contact the students in the classroom and remind them of 
upcoming or missed appointments. 

Some clinics have reported examples of high appointment compliance, for example, in one 
school-based clinic, all the pregnant teenage girls going to term returned for all their prenatal care 
visits and in another clinic, all the students with positive STD tests returned for a final checkup. 

These compliance rates are easy to measure, for the rates are based upon whether patients 
returned to the clinic when appropriate, and this infonnation can be found in the clinic reconls or 
encounter logs. 

However, there may be difficulties in defining compliance. For example, should an 
appointment be treated as a missed appointment if the student is 15 minutes late? A day late? A 
week late? Should a pren^ral care regimen be considered complete if the student initially misses 
one or more appointments, is reminded each time, and then comes eventually for the required 
number of prenatal visits? If a student is scheduled for a clinic visit to refill birth control pills, but 
the student does not come for the appointment because she has stopped having sex, is this 
considered a missed visit? Or is the visit removed from the analysis entirely? 

These questions represent only a small portion of the questions and issues that arise and 
they cannot be resolved in this brief overview. However, at a minimum, evaluators should try to 
select reasonable definitions and criteria and then describe precisely how compliance rates were 
calculated. 

Other types of compliance rates are those based upon patient compliance with recommended 
treatments or behaviors outside the clinic. Examples include: compliance in taking iron if anemic, 
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complice in reducing activity when asthma attacks are imminent, compliance in reducing the 
intake of fats or junk food, and compliance in taking birth control pills. Analysis of this type of 
compliance is usually considered part of an impact evaluation (as opposed tc a process evaluation) 
and thus it is discussed in the impact evaluation section below. 



Student Satisfaction 

Student satisfaction with both the clinic and procedures to use the clinic may significantly 
affect their utilization of the clinic. Thus, assessing that satisfaction may be very useful for clinic 
planning. 

There are at least four different ways to measure student satisfaction. These methods can 
be used singly or in combination. 

First, someone who understands school-based clinic issues and questions and who has 
rapport with youth can interview students in the school. During these interviews, this pereon 
should follow a broad interview outline developed previously with the clinic staff. 

Second, this same interviewer or clinic staff can conduct focus groups with students and 
discuss some of the same issues covered in the interview outline above. Focus groups enable some 
students to speak more freely as they hear other students expressing their views; they also enable 
students to react to comments made by other students. They are especially good for generating 
ideas. On the other hand, because focus groups are not confidential or anonymous, they should 
definitely not be used to elicit personal information about individual behaviors. 

Third, a brief visit assessment can be administered to students as they complete each clinic 
visit. The assessment can include scales measuring satisfaction with different aspects of the visit, 
as well as open-ended questions asking for suggestions or concerns. These assessments can provide 
quick anonymous feedback from clinic users, but, of course, they do not provide information about 
those students who do not use the clinic. 

Finally, questionnaires can be administered to all students in the school (or a representative 
sample thereof). These questionnaires can ask students whether they have ever used the clinic, 
what services they have used, their reasons for using or not using the clinic, their satisfaction with 
the clinic, whether they would recommend the clinic to their peers, and their suggestions for 
increasing their use of the clinic. Such questions may give the most accurate picture of student 
satisfaction. (Examples of these questions are in the Student Health Survey in Appendix A.) 



Quality of Care 

The quality of clinic care can be assessed through site visits by health care providers or 
other professionals experienced with school-based clinic operations who are not directly involved 
with the clinic being evaluated. Site visits can provide more qualitative information than clinic 
records or surveys. Visiting professionals may have a more objective view of the clinic's 
operations and may identify strengths or difficulties with the clinic that clinic staff may not 
recognize or may not express during normal clinic operations. 

Prior to making the site visit the site monitoring team should review a variety of written 
materials, for example, the written goals and objectives, the organizational structure, and tfte 
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budgets. During the site visit, the team should meet with clime administrators and staff 
individually. If possible they should also observe clime operations. 

Although the site visit is a less structured form of assessment, it is still useful to have some 
sort ot guide or checklist for the site team to follow. An example of such a guide is the 
"Reproductive Health and Pregnancy Prevention Inventory" in Appendix C that was developed as 
part of CPO's evaluation project. That guide can be used in site visits to other school- o?^ed 
clinics. 

In addition, the process used to create that guide can be used to create other guides for 
other clinic programs or service components. Several steps should be completed to create such a 
guide: 

o identify potentially important characteristics; 

o have other experts familiar with t ; ifc programmatic area add or revise the initial list; 

o organize these characteristics in a logical manner, 

o have outside experts rate the importance of each characteristic on a 1 to 10 

(or 1 to 5) scale; and 
o calculate the mean, scores for each item. 

Sometimes these guides can then be used to calculate an overall numerical score for a particular 
program; more commonly they suggest programmatic areas that need to be implemented or 
improved. 



Costs 



By simply dividing the clinic's costs for a given time period by the number of encounters 
for that time period, it is possible to estimate the cost per encounter of that clinic. Such estimates 
are important because school-based clinics may provide services more cost-effectively than other 
health providers because of the school-based clinics* location and specialization in adolescent 
concerns. 

Once again, this seemingly easy task is not as easy as it might appear, because estimating 
the true costs of clinics accurately is often difficult. Total costs should include reasonable estimates 
for 1) overhead (even if that overhead occurs in another hospital or department of health); 2) the 
space, utilities, and maintenance of the clinic space (even if that is provided by the school); and 3) 
in-kind services (even if those are provided by other departments or agencies). It is easy to forget 
such costs, and there are, of course, pressures to ignore them, but cost analyses must include these 
costs in order to be complete (though they should be separable, so that monetary can be identified 
if necessary). 

School-based clinic staff often spend considerable time doing outreach in the schools. For 
example, they may perform educational functions such as giving presentations in health education or 
sexuality education classes, or organizing school-wide health fairs or assemblies. It is difficult to 
incorporate these activities when estimating the costs per encounter of clinics. On the one hand, 
ignoring all these activities unfairly increases the apparent cost per encounter. On the other hand, 
counting all the students reached during these group activities unfairly increases the apparent 
number of visits and reduces the cost per encounter. Thus, it may be most accurate simply to treat 
group activities separately-both to describe them separately and to estimate their costs separately. 

Estimates of the cost/encounter are only meaningful when compared with the cost/encounter 
estimates of other service providers nearby. These comparisons should be made with caution and 
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should recognize differences in the populations served, quality of care, type of care, and other 
factors that might significantly affect the cost/encounter estimates of one or more of the providers 
For exampl;, school-based clirJcs serving adolescents may need to spend a greater amount of time 
with each yatient than would clinics serving older adults who are more knowledgeable. School- 
based clinics may also spend more time providing counseling for psycho-social problems Such 
counseling may be very important, but it is also very time consuming and can substantially increase 
the cost per visit 

r C J° T additional information on doing cost-effectiveness or cost-benefit analyses, see Levin's 
Cost-Effectiveness: A Primer. For a thorough analysis of the costs of reproductive services of a 
school-linked clinic, see Zabin's article, "The Baltimore Pregnancy Prevention Program for Urban 
Teenagers: How Much Did It Cost?" * 



V. Conducting an Impact Evaluation 



Impact evaluations are very important, because the goals of many school-based clinics 
reflect many goals of society: to improve the receipt of health care by adolescents, to improve 
adolescent health habits, and to reduce their risk-taking behaviors. These are particularly important 
given that most of the morbidity and mortality of adolescents are caused by risk-taking behaviors 
not by disease. 



The Challenge of Evaluating the Impact 

Although important, evaluating the impact of school-based clinics is not an easy task 
There are several characteristics of clinics that make this task more challenging. 

First, clinics are holistic-they recognize the wide range of youths' needs and they provide a 
wide array of services, but rarely do they provide large numbers or percentages of students with 
any particular service. Thus, clinics may have an important impact upon a small group of students 
in one health area and an important impact upon a different group of students in a different health 
area, but they are less likely to have a large impact upon a large number of students in anv 
specific area. 

Second, most clinics employ a medical model-they focus upon the provision of health care 
to those students who seek them. Typically, they are not funded or staffed to serve all students or 
to reach out to a randomly selected group of students. Thus, evaluation results are likely to reflect 
self-selection factors. 3 



Third, the target population is commonly defined as the students in the school (or a portion 
ot them), and some or many of them may obtain health care elsewhere. Thus, the evaluation can 
only examine the impact of school-based clinics by assessing the incremental impact of providing 
additional services in school-based clinics as opposed to providing whatever services already exist in 
the community. 

Fourth, the fact that the target population consists of students in a school setting raises a 
vanety of ethical and administrative issues. At the very least, it makes it more difficult to use 
expenmental research designs and it means thai research protocols must be approved by the 
appropriate school authorities. 
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h-«;. u l Realise stuteitis are influenced by a very wide range of factors in our society, it is 
difficulv for a school-based clinic to have a very dramatic impact upon participants Thus effects 
are more likely to be small and more difficult to measure. P P Inus - ertects 

Selecting the Goais or Criteria for Evaluation 

The impact evaluation should evaluate the important goals of the clinic. It is neither fair 
nor meamngfiU to measure the impact of clinics upon outcomes that do not reflect important clinic 
objechves. These goals should also be realistic and should take into consideration the Challenges 
discussed above. It is probably not worth the considerable effort of evaluating the impact of a 
program unless there are sufficient resources devoted to the program, the program includes the 
important components necessary to create an impact, a sufficient number of students participate in 
the program, the outcome behavior is amenable to change, and more generally, there is a reasonable 
chance of finding a measurable impact. Finally, the goals should be reasonably specific and well 
denned. Improving overall health" is too general to evaluate rigorously. 

The number of goals that should be evaluated may depend partly upon the magnitude of the 
study. A small study may measure the impact of a clinic program upon only one goal. For 
example, it is fully appropriate to measure the impact of a weight reduction program upon the 
weights of program participants only. On the other hand, larger, more comprehensive evaluations 
of clinics should recognize that clinics provide a wide range of services, and these evaluations 
should not focus solely upon any single goal such as weight reduction. 

The experience of past research on school-based clinics suggests that it is better to try to 
measure the impact of aH the programmatic components on any one outcome criterion than to try to 
measure the independent effect of individual components. In other words, "all the eggs should be 
put in one basket" This is simply because individual components are not likely to have a 
measurable impact: that is why clinics develop multiple components in the first place. 

When conducting an impact evaluation, it is also important to measure dosage or exposure; 
one visit in a clinic program may net have an impact, while ten visits might 

Study Design: Selecting the Focus of the Clinic Evaluation 

A critical question in determining the design of the evaluation is whether to evaluate the 
impact of a clinic program upon the entire student body or only upon those students who actually 
participated in that program. There are good reasons for doing either or both. 
The entire school may be considered the appropriate focus for several reasons: 

o One justification for locating clinics on school campuses is that they have the potential 

fo- reaching all (or nearly all) of the students, 
o When the clinic's programmatic goals are articulated in proposals for funding or 

implementation, many of those goals are stated in terms of the entire student population 

or all the students in need. Less often are they stated in terms of only those students 

who participate in particular clinic programs, 
o Many clinic educational and outreach efforts are directed toward the entire student body 

not toward a selected few students in the clinic, 
o The clinics employ a holistic approach to health care and risk-taking behavior. Such an 

approach assumes or recognizes that some clinic programs may target a particular 

outcome, but other programs may also have an impact on that desired outcome. 

14 
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On the other hand, there is also a very good reason for evaluating the clinic's impact upon only the 
program's participants: * ^ 3 



o Partly because the clinics provide a wide range of health services and have only limited 
staff, they typically serve only small percentages of students in the school for any 
particular health problem. Thus, attempting to measure the impact of a program upon 
the relatively small proportion of the student body who participated in the program by 
measuring the impact upon the entire student body will greatly dilute the measured 
impact and possibly obscure it. 

An example will illustrate this. If 30 overweight students participated in a weight reduction 
program, if all of them were weighed before and after the program, and if fifteen of them lost iC 
pounds, then the loss of weight among all 30 students would probably be measurable (and 
undoubtedly statistically significant) and the program would appropriately be considered effective 
On the other hand, if all the students in the school were weighed before and after the program 
(even though only thirty participated in the program), then the loss of weight would probably not 
be measurable (or significant) and the program would not be considered effective. 

In this particular example, it is clear that conducting both types of evaluation may be 
desirable. The evaluation of the 30 students would indicate that the program was effective among 
many of those that participated, while the evaluation of the entire student body would indicate that 
the program did not serve a sufficiently large percentage of the students to have an impact upon the 
enure student body. r 

More generally, it probably is ideal to measure the impact of the programs upon both the 
participants and upon the entire student body. Unfortunately, it is difficult to do either and it is 
especially difficult to do both. If the impact upon only the participants is measured then the 
resulnng report should make clear that only the known proportion of the student body participated 
in the program and that the program had a much smaller impact upon the entire school. 

Some schools have a very high transfer or dropout rate, for example, more than half of the 
students who are in school during the beginning of one academic year are not there the following 
academic year. When there is such a high turnover, it is difficult to measure the impact of the 
clinic upon the entire student body; it is generally more valid to measure the impact upon only 
those students who use the clinic several times. 

The sections below discuss different designs for evaluating the impact upon participants only 
and upon the entire student body. 



Experimental and Quasi-Experimental Designs 

The best methodological design for assessing the impact of any program or treatment upon 
the participants in that program is the classical experimental design. Ir. includes three basic steps- 
1) randomly assigning subjects to the experimental and control groups; 2) administering the program 
or treatment to the experimental group; and 3) collecting post program data on all the subjects and 
comparing the two groups. 

Random assignment should not be confused with haphazard assignment. In random 
assignment, all participants in the study have an equal preassigned chance of being in the 
experimental or control groups. Random assignment is critically important because it eliminates the 
possibility of self-selection. 
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Self-selecDon can be a major problem in clinic evaluation research. For example, students 
who ucciue to participate in a weight reduction program may have a much greater motivation to 
lose weight and thus a higher probability of losing weight than non-participants. Similarly, students 
who come to a clinic for birth control may be far more motivated to avoid pregnancy and to use 
birth control than students who do not come to the clinic for birth control. 

Not only are these self-selection factors important, they may also operate in unexpected and 
unknown ways. For example, in one large school, sexually active students who had used the clinic 
for birth control were more likely to have used birth control recendy and also more likely to have 
been pregnant than sexually active students who had not used the clinic for birth control The 
reason for this unexpected finding is NOT that getting birth control from the clinic increased their 
nsk of pregnancy. Rather, teens who had previously been pregnant and were still in sci:ool were 
more ikely to want to avoid a second pregnancy and were subsequendy more likely to obtain birth 
control from the clinic. That is, prior pregnancy led to the use of birth control from the clinic and 
obscured the fact that use of birth control from the clinic may have reduced subsequent pregnancy 
To eliminate problems of this type, self-selection effects must be prevented. The random 
assignment in the classical experimental design is the most rigorous way to eliminate self-selection 
effects. 

There are also other valid experimental designs that eliminate self-selection effects, and 
there are quasi-experimental designs which are typically easier to implement bur less valid These 
are well descnbed m the classic text on this subject, Experimental and Quasi-Exbmmental Designs 
for Research by Campbell and Stanley. 

This monograph will focus upon several recommended designs. None of them is optimal, 
but they represent the best compromises between administrative feasibility and ability to produce 
valid evidence for the impact of the programs. 



Measuring the Clinic Impact Upon Program Participants 

Design 1; A classical exp erimental design . In theory, at least, it is possible to use a true 
expenmental design to evaluate the impact of a particular clinic program. For example, if a school 
has a substantial number of overweight students and introduces a nutrition and exercise program, it 
may be feasible to identify overweight students in the school, randomly select half of them to 
participate in the program and then compare over time the weights of those both in and not in the 
program. 

In the area of sexuality, there are at least two ways this classical experimental design can 
be used. The first design measures the effectiveness of an outreach effort, while the second 
measures the impact of a clinic program. 

Example 1: 

i) Students (or classes of students) in the school are randomly assigned to experimental 
and control groups. 

ii) A special educational or outreach program is implemented among the students (or 
classes of students) in the experimental group. 

iii) Outcome measures (e.g., clinic utilization or use of birth control) are subsequendy 
measured with clinic records or questionnaires administered in the classroom. 
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Example 2: 



0 Sexually active students are identified during nealth maintenance exams in the clinic and 
are then randomly assigned to the experimental or control groups. 

ii) Members of the experimental group are then strongly encouraged to participate in . 
intensive clinic program that includes counseling, information on birth control, the 
provision of birth control, and rigorous followup. 

iii) Data on sexual activity, use of birth control, and pregnancy are subsequently obtained 
from interviews in the clinic or from questionnaires. 

Although these classical experimental designs can produce valid evidence for the 
effectiveness of the programs, they raise important ethical and administrative questions and therefore 
they are difficult to implement for several reasons. What happens to students in the control grouos 
who want to participate in the program or who want services? What happens if members of die * 
expenmental group do not want to participate in the program or their parents do not want them to 
participate? 

Some people believe that it is ethically wrong to involve students in such experiments 
Certainly, many school administrators and parents do not want "experiments being done on our 
kids." Similarly, many clinic practitioners and others are strongly opposed to denying clinic 
services to students in the control groups who seek such services. 

Unfortunately, there are no easy answers or solutions to these concerns. However, some of 
these concerns can be partially alleviated or countered in several ways: 

o First, students assigned to the control group can be placed on a "waiting list" and then 
transferred to the treatment group after needed data are collected. 

o Second, the evaluation can be conducted during a limited time period, such as the first 
year of the program only. After that time period, all students can participate. 

o Third, if students in the experimental group participate in a pregnancy prevention 
program, then students assigned to the control groups can be allowed to participate in a 
drug prevention program (or some other type of program) which is beneficial, but is not 
likely to have an impact upon pregnancy prevention. 

o Fourth, if resources and staffing are limited (and they normally are), then it can be 
recognized that not all students can participate anyway. Thus, Lhe principle of 
randomness instead of the principle of "first-come, first-serve" determines who 
participates in the program. This principle is not necessarily counter to our values of 
equality of opportunity. 

o Fifth, not all -students in the school need to be assigned to the experimental or control 
groups. Thus, a small number of high-risk students could be allowed to participate in 
the clinic program regardless of their being in the experimental or control group. In 
order to maintain the equality of the experimental and control groups, such high-risk 
students can be allowed to participate in the program but during data collection, they 
must be excluded from JjoJh the experimental and control groups. Moreover, allowing 
certain categories of students to be excluded from the study can be done only on a 
limited basis. Otherwise, the ability to generalize from the study would be 
compromised. 
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o Finally, students in the control groups can, of course, continue to seek services from 
other community providers that exist independently and prior to the clinic services. 

Despite these various ways to make this design more palatable, in some communities, in 
some clinics, and for some types of programs, it would still be unethical or administratively 
impossible to implement this design. On the other hand, from a methodological standpoint, it is 
clearly the most valid design, and when the research is important and when this design can 
ethically and administratively be employed, then it should be given serious consideration. 

The rigor of this design would be reduced, but the administrative feasibility of the design 
would be enhanced if design requirements ere relaxed somewhat. For example, the clinic could 
randomly assign students to the experimental and control groups and could rigorously recruit the 
members of the experimental group, while nevertheless still allowing control group students to use 
the clinic services if they wanted to on their own initiative. If only a small percentage of students 
knew of these new services and wanted to use them, the rigor of the design would not be heavily 
sacrificed. The measured impact of the program would be slightly reduced, but this would be a 
conservative bias. Of course, if a substantial percentage of students in the control group were 
given services, then the validity of the evaluation would be sacrificed. 

The inability to assign people randomly to experimental and control groups is a common 
problem in evaluation research; there are two common alternative solutions. First, some researchers 
employ a case/control design in which they try to find control subjects who match the experimental 
subjects on important characteristics. Such control subjects might be the best friends of the 
experimental subjects or other students in the clinic who have the same background characteristics. 
This evaluation design may be valid in some types of outcome research, but less so wherever 
motivation plays a major role in the outcome. Motivation obviously plays a very important role in 
the use of birth control, and thus this is not likely to be a valid desigi. for pregnancy prevention 
programs. 

Second, sometimes researchers try to eliminate the impact of differences between groups by 
identifying those factors which might produce an impact, measuring them, and then statistically 
controlling for them For example, when comparing students who use a clinic for birth control 
with students who u / not, a researcher might statistically control for age, race, time since first 
intercourse, and prior use of birth control. This design is not recommended, however, because 
motivation continues to be a confounding factor. 



Contamination of experi mental and control groups . When experimental designs are used 
within school settings, there is the possibility that the clinic program will effectively change the 
behavior of the members of the experimental group and that they in turn will affect the behavior of 
the control group. For example, if counseling and birth control services in the clinic cause 
memoes of the experimental group to have unprotected sexual intercourse less frequently, then peer 
group norms in the school might change and members of the control group might also engage in 
unprotected sexual intercourse less frequently. This cross-over or contamination effect is especially 
likely to take place when members of the experimental group are dating or going out with members 
of the control group. 

Although this contamination effect is a problem that should be recognized, it should not be 
given undue importance. Few programs have a major impact upon the experimental group and it is 
unlikely that a small change in the experimental group will have a significant impact upon the 
control group-adolescent behavior is a function of too many things and is simply too difficult to 
change for this to be likely to happen. 
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However, this potential problem can be reduced by including in the experimental program 
only a small proportion of the students in the school. If only a small proportion of the students 
participate, then the participants are especially unlikely to affect the behavior of the larger majority 
of students. If reducing the proportion of students that participates unduly reduces the needed 
sample size, then it -nay be possible to replicate the program in other schools, to combine the 
experimental groups for statistical purposes, and thereby to increase the sample size. 

Following up study participants and collecting post program data. As discussed above, 
after students have been randomly assigned to" the experimental and control groups and after those 
students in the experimental group have begun their participation in the program, outcome data 
must be collected from the expen^iental and control groups. When this data collection should 
begin and end depends upon a variety of factors, including when the program is likely to begin 
having the desired impact, how long that impact is likely to enine, and how feasible it is that data 
can be collected. As a general guideline, data collection on outcome measures should begin as 
soon as the program is likely to have an effect and should continue as long as that effect is likely 
to be maintained, assuming, of course, that it is possible to do so. 

Many school-based clinic programs are designed to have an impact upon students primarily 
when students are actually receiving the clinic services and participating in the clinic prog-am. In 
the evaluation of those programs, it is clearly important to begin collecting outcome data while the 
students are still participating in the program. Similarly, if a clinic begins providing family 
planning services or implements a procedure for more effectively tracking family planning patients, 
contraceptive continuation data should be collected while the students are receiving those family 
planning services. 

Other clinic programs are designed to have an impact upon students primarily after 
completion of the program. For example, outreach efforts or educational programs in the classroom 
may be designed to affect subsequent knowledge, attitudes, or behaviors and those outcomes should 
be measured after the outreach or educational interventions arc completed. 

Many outcomes should be measured not just once, but periodically for months or even a 
year or two after participation in the program. However, again this varies with the outcome. Some 
programs may have short-term effects that quickly dissipate. For example, educational programs 
often produce prompt increases in knowledge, but students also promptly forget much of that 
material and the impact of educational interventions upon knowledge may not endure beyond six 
months and may not warrant the cost of measuring it after six months. 

Other programs have effects thsu do not become apparent for many months and longer-term 
evaluations need to be conducted to evaluate these outcomes. For exampie, evaluations of programs 
to delay Jie initiation of sexual intercourse need to measure the program's impact for at least one 
year and preferably for two or more years, because relatively small percentages of students in either 
the treatment or control groups are likely to initiate sex each year, and the difference between those 
percentages is not likely to be statistically significant for one or more years. Similarly, evaluations 
of programs to reduce pregnancy should collect data for at least one to two years after the program, 
because relatively few pregnancies will occur in either the experimental or control groups in a 
shorter period of time. 

Whenever data must be collected, the potential problem of attrition arises. This can be a 
problem even when data are collected during participation in the program, because some members 
of the experimental group may drop out of the program even before the program is completed. 
However., this potential problem becomes particularly severe when follow-up data must be collected 
for a year or more after the program. Students who cannot be reached for follow-up may differ 
from those who can be reached. For example, they may be more transient, less motivated, more 
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way related to the outcome variable. Thus, it is important to reduce attrition as much as possible. 

A widely accepted criterion is that at least 80% of the subjects originally randomly assigned 
\ f T f x ^ nmmv>1 ^ control groups should provide outcome data whenever that data is collected 
If the follow-up rate is less than 80%, then the evaluation should demonstrate that (1) those who 
dropped out of the study do not differ significantly from those who remained in the study or (2) 
the impact of attrition wiU affect both the experimental and control groups equally. 

Following up such a high percentage of study participants can be difficult, especially if 
many of the participants have left the school. There are several methods of improving follow-up 
First, develop a good relationship between the students and both the program and the orogram 
evaluators. Second, during the first collection of data, request from the students the names 
addresses and phone numbers of three people (such as relatives) who will always know where they 
are. Third, contact each study participant every six months and update their address and phone 
numbers. Finally, pay them five dollars to complete questionnaires mailed to them. 

. Design 2: Quasi-experime ntal time-series designs . In addition to the classical 
expenmental design, there are other less valid, but nevertheless useful, quasi-experimental designs 
that may be used. If the outcome measure is already being collected by the clinic, then the time- 
senes design is particularly easy to implement In this design, data are coUected for several time 
intervals both before and after the implementation or change of a program. 

Example 1: 

i) The numbers of new patients coming to the clinic for the first time are counted for each 
month for a year. 

ii) A new program for reaching out to students and encouraging them to use the clinic is 
implemented. 

iii) The numbers of new patients coming to the clinic each month are counted for each 
month for the following year. 

If there is a sudden increase in new patients after the new outreach program is implemented, this 
increase suggests that the program was effective. 

This same design can easily be used to measure other new program elements. For example 
if the interview protocols are changed so that staff automatically ask more questions about 
unprotected sexual activity during every visit, the increase in family planning patients can be 
measured; if the clinic implements a more comprehensive system for tracking family planning 
patients for subsequent visits, the increase in contraceptive continuation can be measured. 

This design has one major weakness. The number of new patients coming to the clinic as 
a result of the outreach (or the increase in contraceptive continuation) must be distinguished from 
lmprovements-either gradual or sudden-that may have occurred anyway. When possible this 
threat to validity should be compiled by having a comparison group. (If the control group is 
randomly assigned, then this d< sijm is a classical experimental design discussed above ) 



Measuring Clinic Impact Upon the Entire School 

Because it is normally impossible to randomly determine whether clinics are implemented 
in specific schools, there are no true experimental designs that can be used to measure the impact 
of clinics upon the entire school. However, there are two quasi-experimental desigrs that are 
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recommended for measuring the impact of the clinic upon the entire school. The design which is 
easier to implement is discussed first. * 

n Design 1: A pre/post cross-sectional ouasi-e xperimental design . In this design, data are 
collected from both the clinic school and the comparison school both before and after the clinic 
opens In the language of Campbell and Stanley, this design is an unmatched pre/post non- 
equivalent control group desiga For each data collection period, the data are collected either from 
all the students in the schools or from large (preferably random) samples of students in the schools 
Whue this design will work with any type of data collected, usually survev data are collected. For 
example, a health survey could be administered to all (or most) of the students in both the clinic 
and comparison schools both before and after the clinic opened. When analyzing the data, changes 
over time in the clinic school are compared with those changes in the comparison school. 

Normally, clinics see far too few students for any particular health outcome during any 
short penod of time to have a measurable impact upon the entire school. Thus, the period of time 
needs to be quite long, such as two or more years. Especially in the area of reproductive health, 
any school clinic is not likely to have a measurable impact upon the entire school in less than two 
years, and because school clinics typically serve smaller numbers of students during their first vears. 
it may be necessary to collect data for three or more years. 

One methodological implication of this is that the baseline data needs to be collected one 
year and the post-clinic data the following years. When administering surveys in successive years, 
it is very important to administer them at the same time each year, preferably in the spring If the 
quesuonnaires are not administered at the same time, then biases may result-students who attend in 
the fall may drop out during the year and not complete a questionnaire in the spring. If the 
questionnaire is administered each fall, the previous spring's seniors who were exposed to the clinic 
the longest penod of time will have graduated and not be available to complete the questionnaire 
Instead, a new class of entering freshman who have not been exposed to the clinic will complete 
the questionnaire. In contrast, if it is administered at the end of the spring semester, all the classes 
of students will have been exposed to the clinic for at least two semesters. 

It should be fully recognized that this is not the typical pretest/posttest design. That is, the 
students who complete the baseline survey are not exactly the same students who complete the post 
surveys. Rather, the baseline and post ^rveys provide two (or more) "snapshots" of the entire 
school at two (or more) points in time. Nevertheless, the changes in the "snapshots" in one school 
over time can be compared with the changes in the other school. 

The major problem with this design is that factors other than the implementation of the 
clinic may produce changes in the students' behavior. If the factors are community wide, then they 
may affect both the clinic school and the comparison school equally. If so, then the impact of that 
factor would be statistically controlled by comparing the change in the clinic with that in the 
comparison school. However, if the factors affect oniy the clinic school (or if a different set of 
factors affect only the comparison school) then the analysis may be invalid and changes (either 
positive or negative) may be incorrectly attributed to the clinic. 

Unfortunately, there are a wide range of factors that can affect schools. First, the student 
population can change, particularly if the school boundaries change, a magnet school program is 
implemented, or the community changes. Second, the school can implement other programs which 
may affect outcomes (a system to automatically call parents when their children are absent, for 
example). Third, other programs in the community may change-a health or family planning clinic 
may open or close nearby. Thus, when using this design, it is important to examine as many of 
these alternative hypotheses as possible, before concluding that the clinic did or did not have w 
impact. 
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Frankly, it is often very difficult to administer surveys in the clinic and comparison schools 
both before the clinic opens and again two or more years later. Often clinics open without 
administering a questionnaire before the clinic opens, thereby eliminating the possibility of obtaining 
baseline data. If this occurs, the best remaining possibUity is simply to use a maximally similar 
school nearby as the basis for comparison and to measure with the health survey all background 
variables that may be related to the outcomes variables. These background variables should then be 
statistically controlled with multiple regression or other appropriate statistical techniques. 

Alternatively, if the survey is administered in both the clinic and comparison schools in the 
fall, then the ninth graders would not have used the clinic and they can be treated as baseline data. 
More precisely, the difference between the ninth graders and the upperclassmen in the clinic school 
can be compared with the corresponding difference in the comparison schooL 

Sometimes it is impossible to gain the approval needed to survey a nearby school without a 
clinic. If this occurs, the post-clinic survey in the clinic school should simply be compared with 
the pre-clinic survey. In this design, it is particularly important to explore other changes that took 
place in the schools or communities and may have affected the students behavior. 

Neither of these two more limited designs is as valid as a design which includes both an 
expenmental and control group and baseline and post data. And neither of them can measure with 
any certainty any small changes that the clinic programs may have had. However, both of them 
can provide useful information on major changes produced by the clinic, especially if multiple types 
of data are gathered and if important background variables are measured and statistically controlled. 

Design 2: A longitudinal quasi-experimental design . In the cross-sectional design above, 
"snapshots" of different student populations are obtained. Sometimes it is possible to use a 
longitudinal design and to conduct successive surveys of the same students. In this longitudinal 
design, students entering the school are surveyed before they use the clinic and then annually for 
two or more years afterward. The changes over time in individual students in the clinic school are 
then compared with die changes over time in students in comparison schools. A follow-up rate 
(percentage of students who participate in the first wave of data collection who also participate in 
the second wave) of 80% at two years is necessary in order to feel confident that study findings 
represent changes in the population under study. 

This design has the same limitation as the design above: namely, there may be many 
differences other than the clinic between the clinic school and the experimental school. These 
differences can be controlled statistically, to the extent that they are measured. 

The real richness of this design is provided by treating data as confidential (instead of 
anonymous) and by combining data from the surveys with clinic and school data. Thus, more 
complete histories of each student can be created. These data can become particularly rich if in- 
person iaterviews (instead of questionnaires) are used and important topics are explored in-depth. 
For example, use of the clinic for birth control and subsequent use of birth control can be more 
fully discussed. Although it is possible to conduct in-person interviews in a pre/post cross-sectional 
design, the additional benefits of in-person interviews are rarely worth the additional cost in that 
design. 

Another advantage of this design is that the tracking of students enables the researchers to 
interview students who dropped out of school (because of pregnancy or other reasons). This 
eliminates one bias of the cross-sectional design which only includes students still in school. 

This longitudinal design is very difficult to implement and very costly for several reasons. 
First, all students in the sample must be tracked over two or more years, and tracking them is a 
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difficult and time consuming process. Second, the loss of anonymity typically means that parental 
consent must be obtained, and obtaining parental consent is very difficult school-wide. Third, to 
create a rich data base, personal interviews have to be conducted, They are far more time 
consuming that surveys in the classroom. 

In sum, this design can produce very rich data, but it should only oe undertaken with 
experienced researchers with considerable funding, and it still cannot overcome the basic limitations 
of quasi-experimental designs. 

VI. Selecting A Sample 

Samples of students (or other groups) arc especially likely to be employed in needs 
assessments and in impact evaluations. Because many process statistics-for example, the number of 
patients served and the number of encounters-are based upon all students served (and not a sample 
of patients served), samples are less commonly selected in process evaluations. However, some 
process analyses such as satisfaction surveys may also employ samples. 

Given a defined population of people, there are two important characteristics of any sample 
selected from that population-its representativeness and its size. Although both characteristics are 
important, it is normally more important that the sample be representative than large. That is, a 
small representative sample is usually better rhan a large non-representative sample. 

Every reasonable effort should be made to select a representative sample; otherwise biases 
can easily enter in. An excellent method of selecting students is to use random assignment of 
students (or classrooms). Once a random sample of students (or classrooms) is selected, then it is 
important that nearly all the members of this sample actually provide the data needed. 

When collecting data in the classroom, it is difficult or impossible to randomly select 
individual students; instead, entire classrooms can be randomly selected. However, this must be 
done cautiously. Especially when the number of classrooms in the school is not large, chance 
alone may cause there to be too many stuuents who are in above (or below) average classes. Thus, 
one should select a stratified sample of classes, controlling for the general level of the classes and 
any other important class characteristics. 

Given a representative sample, increasing sample size increases the precision of the 
estimate-large samples produce more precise estimates than small samples. 

When conducting needs assessments, estimates usually do not have to be very precise. For 
example, when deciding whether or not to provide service, it usually does not make much 
difference whether 40% or only 35% of the students have not seen a doctor during the previous 12 
months. Thus, relatively small samples of only a few hundred students may be sufficient. On the 
other hand, if more precise estimates arc needed (for example, if it is important to know that the 
pregnancy rate is 9% and not 11%), then larger samples should be used. 

When conducting impact evaluations, programmatic changes in behavior are likely to be 
small, and thus very large samples _nay have to be used to detect a programmatic change. That is, 
even if a program has a small but programmatically important impact upon the participants, that 
impact may not appear statistically significant and the researcher may incorrectly conclude that the 
program had no impact unless the sample size is large. 

For example, if a school clinic successfully reduces the pregnancy rate in a school by 20% 
by reducing it from 20% to 16%, this should be considered a substantial success. However, that is 
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actually an absolute decrease of only 4% (20%-16%=4%), and in order for there to be an 80% 
chance of finding this 4% drop to be statistically significant at the .05 level, about 1 523 
respondents need to be in the pretest sample and an equal number in the posttest sample If fewer 
respondents completed the survey, the chances of finding results to be statistically significant would 
be even smaller, even if the program really did have such an impact 

In Table 1 are examples of approximate sample sizes required in order to have a chosen 
probability of finding actual changes to be statistically significant. Note that if the clinic reduces 
the pregnancy rate (or any other proportion) by more than 20%, then the needed sample size drops 



r able 1 

Examples of Approximate Sample Sizes Needed to Find Given 
DMerences In Proportions To Be Statistically Significant 1 

Desired Probability of 
Sample 1 Sample 2 Finding Difference To Be Needed 
Proportion Proportion Statistically Significant Sample size 2 



.40 


.35 


.80 


1535 


.40 


.35 


.50 


751 


.40 


.30 


.80 


370 


.40 


.30 


.50 


181 


.40 


.20 


.80 


83 


.40 


.20 


.50 


41 


.20 


.16 


.80 


1523 


.20 


.16 


.50 


745 


.20 


.15 


.80 


1193 


.20 


.15 


.50 


584 


.20 


.10 


.80 


210 


.20 


.10 


.50 


103 



2 All examples assume that all tests of significance are two-tailed. 

This is the ssmple size needed for each sample (e.g., each pretest and posttest 
sample or each experimental and control sample). 

Example 1: If 40% of the males in the treatment group used a condom the last 
time they had sex, but only 30% of the males in the control group used a condtm, 181 
males would have to be included in each group in order for this difference to fo<ve a 50% 
chance of being statistically significant 

Example 2: If 20% of the females in a school became pregnant each year before a 
clinic opened and only 15% became pregnant each year after the clinic opened, 1 193 
females would have to be sampled before the clinic opened and another 1193 females 
would have to be sampled after the clinic opened in order to have an 80% chance of 
finding this improvement to be statistically significant 
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Table 2 

Recommended Outcome Measures 
In the Area of Sexual Activity 



Outcome Measures 

Sexual Activity 

Ever had sex 

Age of first intercourse 

Date of first intercourse 1 

Frequency of intercourse in last four weeks 

Use of Birth Control 

Ever used birth control 

Ever been to a clinic for birth control 

Date of first use of birth control pills 1 

Use of birth control during first intercourse 

Type of birth control used during first intercourse 

Use of birth control during last intercourse 

Type of birth control used during last intercourse 

Pregnancy and Pregnancy Outcome 2 
Ever pregnant 

Number of times ever pregnant 

Resolution of last pregnancy 

Date of resolution of last pregnancy 

Other Important Measures 

Reasons for never having had sex 
Reasons for using birth control 
Reasons for not using birth control 
Where contraception is most often obtained 



The number of months between first intercourse and first use of birth control pills can be 
used as an indicator of unprotected sexual activity. 

Equivalent questions should be included for males. 
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Table 3 

Suggested Outcome Measures in Other Areas 



Receipt of Health Care 

Whether immunizations are up to date 

Number of months since student had a physical exam with blood test, urine test, and blood 

pressure checked 
Number of years since hearing and vision were checked 
Number of months since student visited a dentist 

Whether student did not receive health care needed during previous 12 months 
Number of times doctor visited 

Number of times treated at an emergency room or hospital and reasons for such use 
Number of nights spent in a hospital 

Health-Related Habits 
Nutrition 

Frequency of eating three meals a day 

Frequency of eating proper portions of each of the food groups 

Frequency of eating "junk" food low in nutrition 

Frequency of eating fried foods 

Frequency of going on severe diets 

Frequency of intentional vomiting to reduce weight 

Exercise 

Frequency of getting 20 minutes of vigorous exercise at least three times 
each week 

Dental 

frequency of bmshing teeth 
Frequency of flossing teeth 
Frequency of visiting a dentist 

Substance use 

Frequency of alcohol consumption in life 

Frequency of alcohol consumption in past month 

Number of drinks of alcohol consumed at one time 

Ever chewed tobacco 

Frequency of chewing tobacco 

Ever smoked cigarettes 

Frequency of smoking cigarettes 

Ever smoked marijuana 

Frequency of smoking marijuana 

Ever used other illegal drugs 
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Safety 

Frequency of wearing a helmet when riding a bicycle 

Frequency of driving too fast 

Frequency of riding in a car going too fast 

Frequency of driving while influenced by drugs or alcohol 

Frequency of being in a car with a driver influenced by drugs or alcohol 

Frequency of using seat belts while riding in a car or truck 

Frequency of riding a motorcycle 

Frequency of wearing a helmet when riding a motorcycle 

Frequency of swimming in an unsupervised area 

Violence 

Frequency of carrying a knife or gun to school 
Frequency of getting in a physical fight 

Sleep 

Frequency of getting adequate sleep each night 

Health Status 

Self perception of health 

Frequency and severity of stress and frustration 

Frequency and severity of depression 

Frequency of thoughts about suicide 

Ever attempted to commit suicide 

Existence of any serious personal, emotional, or behavioral problems 

Number and types of health concerns 

Height/weight percentiles 

Perception of being overweight 

Perception of being underweight 

Number of days of school missed due to illness 

Ability to perform appropriate physical fitness tests 

School Success/Failure 

Number of days absent from school 
Grade point average 
School plans 

Ever dropped out of school 
Ever in trouble in school 
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VIII. Sources and Types of Data 
for Needs Assessments and Impact Evaluations 



There are a variety of different sources of data that can be uszd to identify need or to 
evaluate the impact of clinics: clinic records, student surveys, school records, and public records 
Separately and in combination they can produce a substantial amount of evidence for the 
effectiveness of clinics. 



Clinic Records 



As discussed above, clinic records can produce a vast arr»v of process statistics on clinic 
utilization. Some of this process data can be used to determine what services students received, 
and whether students received single services or clusters of services that have the potential for 
reinforcing each other. If these data for specific individuals are linked with identification numbers 
to outcome data, then students can be divided into those receiving no interventions, single 
interventions, and multiple interventions. 

Clinic records can also provide extensive outcome data, for example, on contraceptive 
continuation. In addition, as clinic staff complete each student's medical history, they can ask the 
questions necessary to provide data on nearly all the measures in Table 2. (The CPO SCMIS 
system provides information on sexual activity, use of birth control, pregnancies and births.) 

In the area of reproductive health, clinic records are best suited to providing more reliable 
data on contraceptive continuation. Unfortunately, there are many questions that need to be 
resolved before measuring contraceptive continuation. How should a patient who has probablv 
stopped having sex be treated? Is that person removed from the analysis, counted as complying, or 
counted as not complying? When determining a twelve-month contraceptive continuation rate, how 
should a student be treated who stopped using birth control for a month, but then started to use i: 
again? Research has demonstrated that compliance rates depend greatly upon how compliance is 
defined. These issues cannot be resolved in this monograph, but they are more fully discussed in 
Clinic-Based Methods of Measuring Adolescent Sexual Activity, Contraceptive Continuation, and 
Sexual Risk-taking" by Kirby and Hera. 



Health Surveys 

Questionnaires can incorporate a very wide range of behavioral and health measures. For 
example, they can measure: 

o receipt of medical care 

o elapsed time since getting medical care 

o use of emergency medical facilities and reasons for such use 

o number of nights spent in a hospital 

o depression 

o safety habits 

o nutritional habits 

o use of drugs and alcohol 

o extent of sexual activity, use of birth control, and pregnancy history 
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Questionnaire design. There are many texts on questionnaire design and their innumerable 
suggestions cannot be summarized here. Zabin and Hirsch give many useful suggestions for 
questionnaire construction and describe their questionnaire. In Appendix A is a recommended 
health survey that includes the questions needed to provide data on all the pregnancy prevention 
measures in Table 2 and many of the measurer in Table 3. 

When designing questionnaires, the reliability and validity of the questions should be given 
considerable attention. Reliability is the consistency with which items on the survey instrument are 
answered the same way when asked more than once. Research has demonstrated that most items 
included in the Student Health Survey in Appendix A are reliable. 

The validity of a question is the degree to which it is really uieasaring what the researcher 
wants it to measure. Sometimes this is related to issues of comprehension of the items. 
"Incorrect" answers may be due to the respondents' misunderstanding of the question. Because 
many of the questions such as "How old are you?" and "If you were sick and needed medical care f 
where would you go?" are straightforward, face validity can be used as a criterion for determining 
the validity of most items. 

In addition to the underreporting of certain events due to misunderstanding of the question, 
there may be underreporting due to the student's perception of the social desirability of the 
behavior in question. For example, students may underrcport behavior on pregnancy, abortion, or 
illegal substance abuse. However, experience has indicated that when questionnaires are properly 
administered with assurances of anonymity, students do not substantially underreport the frequency 
of sexual intercourse or the use of birth control. 

It is always tempting to ask too many questions in a questionnaire, because there are so 
many topics that researchers like to address, but the questionnaire should be limited so that students 
can complete it in about 30 minutes or less. Experience has demonstrated that many students are 
quite interested in questions about their health and sexuality and consequently are willing to 
complete carefully questionnaires that arc longer than questionnaires on other topics. 

If the number of questions that must be asked exceeds the limits of a single questionnaire, 
then two versions can be prepared. The two versions would have the most important questions in 
common, but each would have different sets of less important questions. The two versions can 
then be administered randomly to the students. 

Many schools have special classes of students with low reading abilities; this raises the 
issue of whether or not to administer the survey to them. Some researchers report that students in 
special reading classes are able to answer questionnaires reliably, even when their teachers suggest 
that the questionnaires are too difficult This experience argues for pilot testing the survey with the 
students in question and verifying their ability to complete it Critically, whatever decision is made 
about these students should apply equally to both the experimental and control groups. 

Pretesting the questionnaire. It is always wise to pretest repeatedly successive versions of 
the questionnaire with small groups of students representative of the school. During these pretests, 
students can complete the questionnaire and circle any word or question that was not completely 
clear to them. After the students complete the questionnaire, the researcher should discuss with 
them any possible changes and any questions they may not have understood/ When students are 
paid a nominal amount of money, they are usually happy to do this. 

Obtaining approval. Usually obtaining school approval is not required when conducting a 
process evaluation based upon clinic records, because those records are kept anyway, and analyses 
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of those records are considered a normal pan of clinic operation. However, when needs 
assessments or impact evaluations require the collection of additional data through student surveys, 
then approval from the appropriate school authorities is necessary. 

Different communities and different administrators have very different standards for 
approval, and, of course, who should provide the approval depends at least partly upon the content 
of the questionnaire. Almost always, the approval of the principal must be obtained. Sometimes 
but not always, it is wise also to obtain the approval of the superintendent of schools and the board 
of education. 

Sometimes this approval is difficult to obtain, especially from the schools, because the 
principals may feel that they have little to gain and much to lose by the survey. For example, 
principals may be concerned that the evaluation activities may place an additional burden on 
teachers and students, that some parents may react negatively to their children participating in the 
survey-especially if personal and sensitive questions are asked, or that school-wide statistics on 
various risk-taking behaviors may paint their school in a negative light and diminish its positive 
reputation. 

To gain the cooperation of the school, the value of the survey to die school-the fact that 
the study will help improve the clinic and thereby improve the health of the students should be 
emphasized. And, of course, each of the principal's concerns need to be properly addressed The 
following points should be emphasized as appropriate: 

o The evaluation will require only limited class time-typically only one class period, 
o Most students seem to enjoy and learn from the health survey. It may make them 

more conscious of their risk-taking behaviors that they should address, 
o Parents will be properly notified, and if they do not want their children to 

participate, those children will not participate, 
o All data will be anonymous (or confidential if identification numbers are being 

used). 

o School-wide data will not be publicly released without the approval of the school, 

and if the school does not grant such approval, then the school will not be 

identified in professional publications, 
o The collected data will be used for planning and improving the program to meet the 

identified needs of the students, 
o Similar research has been conducted in many other schools throughout the country 

without detriment to the school. 

In addition, the researcher should be prepared to make some modifications in the design of the 
study, the administration of the survey and the content of the survey. It is important to know what 
components of the study are truly critical and to be able to defend those elements in a compelling 
manner. 

Often questions about sexual behavior trigger the greatest concern. However, if Jie clinic's 
programs place considerable emphasis upon reproductive health, then f -cry reasonable effort should 
be made to include needed questions in the survey. 

These questions are less likely to trigger concern if they are not the sole focus of the 
survey, but instead are part of a broader, more comprehensive health survey. However, if questions 
about sexual behavior continue to pose insurmountable problems and are not critical to the study, 
then they should be dropped and other portions of the health survey administered. 
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AU parents should be notified in writing of the impending survey. The letter should 
describe the contents of the questionnaires accurately, emphasize the voluntary nature of the survey 
discuss the procedures to be used to maintain anonymity, and provide a phone number for the 
parents to call if they have questions or concerns. Finally, the letter should provide some meuiod 
of allowing tne parents to excuse their child from taking the survey-either a phone number to call 
or a letter to be signed and returned to the school. 

Although signed parental consent better assures parent consent and better protects the 
evaluation and the school from subsequent concern expressed by the parents, obtaining such signed 
consent is likely to distort the sampling. Undoubtedly many students will not return a signed form 
and thus will not be able to complete the questionnaire. These students probably differ from 
students who do return the signed parental consent form; thus, excluding them would introduce a 
bias. (In some designs, this bias is offset by the same Has in the control group, and thus may 
pose less of a problem.) ' 

There are a variety of ways to improve the percentage of students providing parental 
consent. A most imponant step involves getting the teachers to recognize the importance of the 
survey and to make every reasonable effort to obtain consent Several researchers have observed 
that when teachers really make an effort to have their students return the parental consent forms, 
then those forms are liUly to be returned. If teachers are not motivated by the compelling nature 
of the study, then incentives may help. For example, they can be given stipends to be used on 
instructional materials in their classroom. 

Incentives can also be used directly with the student*, for example, they can be given small 
gift certificates or ackers to a school dance or game. Alternatively, classroom contents can be held- 
-classes with the highest response rates receive prizes. 

Selecting classes for the survey administration. Some of the issues involving selecting the 
sample of classes have already been addressed above in the section on sampling. However, there 
are two alternative strategies that can also guide the selection of classes and the administration of 
the survey. 

With the preferred method, a single class (such as Fjiglish) or a small set of classes that 
everyone in toe school must take are selected. A small number of survey administrators then 
administer the. survey to those students during all of these classes. If the number of selected 
classes (e.g., English classes) offered during any specific class period exceeds the number of test 
administrators, then the administrators can return the following day and administer the survev to the 
remaining classes. 

This method hat' two major advantages. First, carefully trained and experienced survey 
administrators can administer it to all the students. This is important because many teachers will 
do an excellent job of administering the survey, but some will not, and if the survey is sensitive, 
both the quality of the data and the anonymity of the answers may be jeopardized. Second, this is 
less disruptive to the teachers and the school. The only teachers who are affected are those 
teaching the selected classes and all of their classes are affected equally. 

This method has one disadvantage: sometimes some students are not enrolled in any of the 
selected classes-despite efforts to prevent this-and thus they are missed. 

For the second method, the survey is administered by all teachers during their home room 
period. This approach has two advantages: the survey is completed simultaneously throughout the 
emire school and is over quickly; and students are not missed because they all have homerooms 
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Of course, the disadvantages arc that some of the teachers wiU not do a good job of 
o^d^X? 6 SUrVCy ^ if n ° mer00m is combined with another class period; that class may get 

Ad ministering the questionnaires , if absenteeism is a significant problem in a school then 
the survey should be administered on Tuesdays, Wednesdays, or Thursdays when students are less 
likely to be absent Obviously, the survey should never be administered during a day or during 
class periods when some other special event will prevent students from completing the survey or 
from completing it accurately. If many students fail to complete the survey because of absenteeism 
then, if possible, it should be administered to those students on subsequent days This is esDeciallv' 
important if absent students differ from other students. 

.. ui Su |^ e y ^nistrators must address conflicting goals. On the one hand, they need to collect 
reliable valid data from all or nearly all the selected students. On the other hand, if the 
questionnaire ha' any sensitive and personal questions, survey administrators need to maintain the 
voluntary nature of the survey and the anonymity of responses to any sensitive questions For 
example, it could be quite emotionally harmful to some teenage girls if others found out that they 
were sexually active or had been pregnant Everything within reason must be done to assure the 
anonymity of responses. At a minimum, survey administrators should: 

o Seat students so that it is maximally difficult for them to see each other's 
questionnaires. 

o Explain to students the importance of the study and the steps being taken to assure 
anonymity. 

o Emphasize to students that completing the questionnaire is voluntary, 
o Hand out scratch paper to cover up answers. 

o Hand out pencils with identical lead so that the color of ink will not identify 
respondents. 

o Hand out and collect questionnaires individually, so that students do not walk 

around and so that all questionnaires are collected, 
o Mix up the questionnaires so tha' no one will know whose questionnaire is on too 

or bottom. K 

Coding the data . The questionnaires are, of course, self-reports. Although most students 
complete most questions honestly, an unknown number of students may intentionally distort their 
answers, and some may make honest mistakes. All questionnaires should be examined for such 
distortions, and those that are clearly invalid should be excluded from the analysis. Records should 
be kept of the number of invalid questionnaires and separate analyses or reviews of them should be 
completed if there are many. 

After data are coded and keypunched, they need to be statistically analyzed Statistical 
methods are outside the purview of this monograph; in addition there are already innumerable books 
on statistics. If the evaluators are not familiar with the statistics appropriate for their analysis, then 
they should obtain assistance from a nearby university with such expertise. 

. Using computers to collect data from ai,H«m s Another method of conducting a survey 
either for a needs assessment or possibly for an impact evaluation includes the use of computers 
and computer software. Computer programs can be written which ask each student all the needed 
questions sequentially. The computers and their software are placed strategically in classrooms or 
elsewhere where students can complete the computer survey. 

This method has several advantages: computers represent an exciting medium for many 
students and may thereby motivate them to complete the survey; the computer program can easily 

33 



ERIC 



40 



be designed to skip unnecessary questions (e.g., if a sntdent has nev., had sex, there is no need to 
ask questions about the use of birth control); all the answers are automatically recoided in a data 
file, thereby eliminating the need for coding and keypunching; and the computer program can 
provide useful information tailored to each student (e.g., those behaviors posing the greatest risk 
and sources of consultation or help). 

On the other hand, using compters also poses several problems: programming tl«: 
computer is more difficult than writing a questionnaire; the number of computers available will 
normally be limited and thus not all students in each class can complete the survey at the same 
time; often it is difficult to have a random sample of students complete surveys on the computer, 
and finally, compuivrs may intimidate some students. 



School Records 

Absenteeism and dropout rates. Schools usually know the number of students who are 
absent each day and also the percentage of days missed during the entire year. Many, but not all 
schools, also know the number of students who have dropped out Dropout rates, however, should 
be compared with caution, for they often vary greatly from one month to another, generally 
increasing during the spring, and different schools have different procedures for counting dropouts. 

Occasionally schools have estimates of the number of girls who dropp*. out because of 
pregnancy, but these estimates arc very unreliable, underestimating the true number of pregnancies. 



Sometimes schools do have accurate information on the number of girls that transferred to a 
special school for pregnant girls. Because these data have already been collected, it is tempting, 
and sometimes useful to observe trends in these data over time. Specifically, it may be useful to 
determine whether these trends demonstrate a relative decline after the clinic opens. 

If pregnancy rates are estimated from the school survey and if a significant number of girls 
did not complete the survey because they had been transferred to a special school for pregnant 
students, then they should also be given the questionnaire to complete and thsir pregnancits should 
be counted as pan of the school pregnancy rate. 



Public Health Records (Merged with School Records) 

Over the years many researchers have tried to estimate the pregnancy rates of high schools 
from public records, but have had little success. Estimates based upon pregnancy rates in census 
tracts have proven to be unreliable, because there is no information about the proportion of the 
pregnancies that occurred among girls who had previously graduated from high school, who had 
dropped out of school, who lived outside the school boundaries, or who attended another school for 
some other reason. There is. however, a reliable method of measuring the birth rate J females in 
school. 



Blf th rates and low-weigh; hjnh rates - By comparing lists of female students enrolled in a 
school with community birth records, it is commonly possible to produce rather accurate annual 
birth rates and low-weight birth rates for the school. Most schools can provide lists of female 
students enrolled for many past years. Hither hospitals, departments of health, or other city or stat< 
agencies house the birth records. 
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Jf 6 ™? ItulI ° n ? whicn house *e birth records may properly be concerned about maintaining 
the confidentiality of the birth records. If so, the evaluator can simply pay that institution to 
complete the protocol itself, and to provide only the annual rates. This maintains the confidentiality 
of the birth records. The feasibility of this method has been demonstrated by researchers who have 
effectively employed it in several cities throughout the country. 

If the number of birth certificates is very large, then the matching should be completed by 
computer. Such matching is not flawless: when the number of names is large, more than one girl 
may have the same name, the same girl may use slightly different versions of the same name (e g , 
initials instead of full name), or the birth certificate may contain a married name while the school 
records use the maiden name. On the other hand, the number of these errors appears to be rather 
small, and cnucally, they tend to be random errors which are less likely to produce any systematic 



If the number of birth certificates is small (less than a few thousand), then it is probably 
easier and more accurate to do the matching by hand. 

The exact protocol to follow when matching names wiU vary with the community and the 
form in which the data are kept However, because the birth records are usually not in alphabetical 
order, but the school's lists of names usually are in alphabetical order, the following system worses 
well: 



Task I: 

1. Proceed through the birth records. 

2. Check whether the mother was 19 or younger when pregnant. If not, return to 
step 1. 

3. Check whether the mother lived in the appropriate geographic area. If not, return to 
step 1. 

4. Check the mother's name against the alphabetical list of students' names. If the 
name is not there, return to step 1. 

5. If the name is there, check whether the mother's age on the birth record is 
consistent with the mother's date of birth from the school record. Also check other 
background characteristics believed to be accurate (e.g., race). If they do not match, 
return to step 1. 

C. If the name and demographic data match, check whether the estimated date of 

conception overlaps with the school enrollment period. If it does not overlap, return 
to step 1. 

7. If it does overlap, then write the date of birth and the weight of the infant beside 
the name. 

Task II: 

1. For each academic year, count the number of girts who conceived while in school 
that year. For each year, also count the number of low birth weight infants. If 
desired, create breakdowns for each grade (freshmen, sophomore, etc.). Because 
seniors have higher birth rates than students in the lower grades, either 1) define the 
academic year as ending May 31, and beginning June 1, or 2) define the academic 
year as September 1, through August 31, and remove all conceptions occurring 
among seniors after they graduated. 

A major advantage of this birth rate methodology is that it can be completed long after the 
clinic has opened. All that * r.Cvcssary is that the schools have a copy of all the female students 
that were enrolled each semester. School? usually keep such records for many years. And, of 
course, birth records are kept indefinitely. 
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Once the birth rates for each year have been calculated, graphs should be created 
demonstrating the trends over time. If the graphs indicate that the birth rates began to decline 
more rapidly after the clinic opened (allowing for the gestation period), then mc data should be 
statistically analyzed. 

Although few researchers have analyzed these birth rates rigorously, past experience 
demonstrates that birth rates vary considerably from year to year, and unless the clinic dramatically 
reduces the birth rates, it is difficult to separate the effects of the clinic from the normal random 
variation in the birth rates. Furthermore, it is important to calculate birth rates for several years (at 
least five) before and after the clinic opens. 

Other annual rates. In theory, this procedure will woric with any health outcome or health 
condition for which all the names of the people having that condition are centrally located. In 
practice, only birth ecords are usually located in one or a small number of facilities. However, if 
one or a small number of health departments have complete records on cases of STD, then this 
same procedure may work there. Or, if there are only a small number of hospitals that students 
might attend, then their records could be examined for annual changes in hospitalization rates for 
the student population. 

iX. Reporting the Results 

The results of valid process and impact evaluations should always be reported to the 
community and to other professionals regardless of whether those results are positive or negative. 
Obviously, if only positive results are reported, then health professionals will have a biased 
understanding of the effectiveness of school-based clinics. In the long run, such biases do not help 
youth, and helping youth should be the ultimate goal of school-based clinics. 

Before makjig research results public, they should be discussed with the clinic 
administrators and alternative explanations for the findings should be explored and the release of the 
findings should be coordinated with concerned parties. 

Regardless of whether results are positive or negative, the limitations of research should be 
stated clearly. In the real world of research and evaluation, nearly all studies have limitations that 
should be recognized by others when they subsequently design health programs or research. 

The reports should describe the way the programs junction, the services they provide, 
staffing patterns, barriers to implementation, and other topics describing the clinic. 

When reporting the results of impact evaluations, other points should also be made, if they 
are true in any particular case, and especially if the results do not indicate a measurable and 
positive impact upon behavior 

o The amount of health care provided by the clinics is substantial. 

o The value of this care should be fully recognized, even if students might have 

received services elsewhere had the clinic not existed, 
o Changing adolescent risk-taking behavior is very difficult and any improvements 

constitute real success, 
o School-based clinics represent a rather limited intervention and should not be 

expected to have a dramatic impact upen adolescent behavior, 
o The clinics evaluated undoubtedly have a variety of positive effects on selected 

individuals that cannot be measured with most research and statistical techniques. 
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X. Summary 



This monograph describes three basic types of evaluation research that can be used to 
evaluate school-based clinics. In addition, various means of collecting data for each of these types 
was described and discussed in the specific context of school-based clinics. In the following 
paragraphs, the most important points in the monograph are highlighted. 

The most useful evaluations are those in which more than one kind of data is collected 
Needs assessments can help clinics identify the critical needs of the students and then design 
programs to meet those needs. Once those programs are implemented, it is important to know that 
students used the clinics for particular services and it is even more important to know that those 
services had the desired impact 

With student surveys, parent surveys, or community data, student needs can be identified. 
With the OPO SCMIS system or other encounter forms it is relatively easy to collect process data 
on clinic utilization. Student satisfaction with the clinic can also be easily measured and site visits 
by other professionals can help identify areas of strength and areas needing improvement in the 
quality of care provided. 

It is more difficult to measure outcomes. This is particularly true because 1) the clinics 
typically serve relatively small proportions of the students for any particular health outcome and 2) 
the students who choose to go to the clinic differ in many ways from those jvho do not. Thus, 
diffusion and self-selection are major problems. 

The traditional solution for this problem is to use an experimental design and to measure 
the impact of the clinic upon the participants. Such designs should be used whenever possible, and 
more generally, research studies should focus upon the impact of the clinic upon those students who 
are actually served by the clinic, and not upon the entire student population. 

These experimental designs need to be combined with large sample sizes; only then will the 
modest behavioral changes produced by clinics oecome statistically significant 

There are also good reasons to try to measure the impact upon the entire school-after all, 
they are the focus of the clinics. Since it is not always possible to measure that impact with true 
experimental designs, quasi-experimental designs are the second-best alternative. If implemented 
with care, these designs can detect substantial changes produced by the clinics. However, they are 
not likely to measure accurately small changes. 

The best method of measuring the impact of the clinic upon the entire student body is to 
collect survey data from all the students in the school or merge lists of student names with birth 
records or other public records. Rates can then be compared over time or between clinic and 
maximally similar comparison schools. 

It may be unrealistic to expect that clinics will have an impact upon the health outcomes of 
all the students in the school, many of whom are not served in the clinic. It may be more prudent 
and rair to clinics to plan specific evaluations of particular programs and the participants within 
them. 

On a final note, if the methods discussed herein are employed properly, they can help 
clinics identify more accurately students needs, improve their clinic services, and provide more 
accurate documentation for the effectiveness of those services. 
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Appendix A: Student Health Survey 



This survey is being done to help us understand what health services arc most needed by 
the students of this school. We need to know several things about your health and your health 
care. Only you can really tell us about these. 

Filling out this questionnaire is up to you. You do not have to complete it, but you will 
help us a lot if you do. 

Do NOT put your name anywhere on these pages. We want your answers to be secret 
No one (not even your teacher) will know that these answers are yours. 

Write your answers directly on this questionnaire. Do NOT put them on a separate sheet of 

paper. 

Your answers are important. Please answer each question carefully and honestly. 



Thank You 



1. Are you: male female 

2. In what month and year were you bom? 19 

Month Year 

3. What grade are you in? 9 10 11 12 

4. When did you first come to this high school? 

— Fall 1983 _ Spring 1984 _ Fall 1984 
_ Spring 1985 _ Fall 1985 _ Spring 1986 

— Fall 1986 Spring 1987 Fall 1987 

5. What grade do you usually get in your classes? A B C D 

6. Are you: (CHECK ONLY ONE) 

— Black (not Hispanic) White (not Hispanic) 

— Hispanic Asian or Oriental 

American Indian Other 



7. Whom do you live with now? (CHECK ALL THAT APPLY) 

— mother father stepmother 

— stepfather other family members 

— grandmother grandfather other (not family) 

8. How many people live in your house, including you? people 
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9. How far did your father go in school? (CHECK THE HIGHEST LEVEL) 

did not graduate from high school 

graduated from high school 

went to vocational school or got other training 

started college 

graduated from college 

10. How far did your mother go in school? 

(CHECK THE HIGHEST LEVEL) 

did not graduate from high school 

graduated from high school 

went to vocational school 

started college 

graduated from college 

11. Does your father or stepfather have a job? 

yes (Go To Question 11a.) 

no (Go To Question 12.) 

— I don't have a father or stepfather (Go To Question 12.) 

11a. Is his job: full-time part-time 

12. Does your mother or stepmother have a job? 

yes (Go To Question 12a.) 

no (Go To Question 13.) 

— I don't have a mother or stepmother (Go To Question 13.) 
12a. Is her job: full-time part-time 

13. Many families get money from other places. Does anyone in your house get money from 

welfare, social security, or public assistance? 

— yes no don't know 

14. Does anyone in your family get food stamps? 

— yes no don't know 

15. Do you get reduced-price or free lunches at school? 

yes no 

16. What do you think you will do in the future? 

quit high school 

finish high school only 

— finish high school and go to college or get other training 

17. Do you like yourself? 

— all the time usually sometimes 

not very often almost never 

18. How healthy do you think you are? 

— very healthy pretty healthy 

— not very healthy n ot at all healthy 
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19. During the PAST YQ\1R WERKfi, how many days of school did you miss because you were 

sick? days 

20. During the PAST FOUR ^EKfi how many days have you skipped or 

cut school? days 

21. When did you LAST go to a dentist? 

— during the past 12 months about 1-2 years ago 

over 2 years ago never 

I don't remember 

22. When did you LAST go to a doctor? 

— during the past 12 months about 1-2 yeare ago 

over 2 years ago never 

I don't remember 

23. Where did you go the LAST TIME you were seen by a doctor? 

a private doctor's office 

a hospital clinic 

an urgent care clinic 

the health department 

a hospital emergency room 

a military clinic 

other 

24. Did you have to miss school the LAST TIME you saw a doctor? 

yes (Go To Question 24a.) 

no (Go To Question 25.) 

24a. How many class periods did you miss when you went to the doctor 
the LAST TIME? _ class periods 

25. Did either of your parents have to leave work to take you to the doctor the LAST TIME ? 

yes no 

26. At any time during the LAST 12 MONTHS were you ever sick or hurt or have some type of 

health problem so that you needed medical care? 

yes (Go To Question 26a.) 

no (Go To Question 27.) 

26a. Did you always get medical care? 

yes (Go To Question 27.) 

no (Go To Question 26b.) 
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26b. Why didn't you get medical care? (CHECK ALL THAT APPLY.) 

— At the time, I didn't think I really needed to see a doctor. 
— . My parents didn't think I needed to see a doctor. 

— Medical care was not available when I needed it. 
It cost too much money. 

I didn't know where to go. 

— I didn't have a way to get there. 

— The hours were not convenient. 

— I had to wait too long to get an appointment. 
I didn't like the staff. 

— My visit would not be secret 

— They didn't have a special clinic for teenagers. 
I just didn't get around to it 

I was afraid. 

27. During the LAST 12 mo^hs, how many times were you treated at an 

emergency room at a hosp ital? _ times 

28. During the LAST 12 MON^ how many nights, did you stay 

in a hospital? nights 

29. If you were sick and needed medical care, where would you go? 

a private doctor's office 

school health center 

the health department 

an urgent care clinic 

a hospital clinic 

a hospital emergency room 

other 

I don't know 

30. When were you LAST TESTED for 

Hypertension? (high blood pressure) 

— during the past 12 months about 1-2 yeare ago 

— over 2 years ago never 

I don't remember 

Scoliosis? (curved spine) 

— during the past 12 months about 1-2 years ago 

over 2 years ago never 

I don't remember 

Sickle cell? 

— during the past 12 months about 1-2 years ago 

over 2 years ago m never 

I don't remember 

Anemia? (low blood) 

— during the past 12 months about 1-2 years ago 

over 2 years ago never 

I don't remember 
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31. When was the LAST TTME you had: 
A physical exam? 

— during the past 12 months about 1*2 years ago 

— over 2 years ago never 

I don't remember 

Your urine tested? 

— during the past 12 months about 1-2 years ago 

over 2 years ago never 

I don't remember 

Your blood tested? 

„ during the past 12 months about 1-2 years ago 

over 2 years ago never 

I don't remember 

FOR FEMALES ONLY: A breast exam? 

— during the past 12 months about 1-2 yeare ago 

over 2 years ago never 

I don't remember 

FOR FEMALES ONLY: A pelvic exam? (female exam) 

— during the past 12 months about 1-2 years ago 

over 2 years ago never 

I don't remember 

32. During the LAST SCHOOL yf^R, did you ever feel that life was not worth living? 

— no yes, sometimes yes, often 

33. During the LAST SCHOOL YfiAR r did you have thoughts of ending your life? 

— no yes, sometimes yes, often 

34. Do you have friends or relatives that you can turn to for help 

wnen something is troubling you? 

— no yes, sometimes yes, often 



We would like to ask you some questions that are personal and can be difficult to answer 
ALL of your answers will be kept secret 

Because we are concerned about teenage pregnancy, in this survey the words "had sex" 
mean that a female and a male have "made love," "done it," or "gone all the way." That is, "had 
sex" means'that a male's penis was in a female's vagina. 

35. Some teenagers have had sex and others have not Have you ever had sex? 

yes (Go to Question 36.) 

no (Go To Question 53.) 

36. How old were you the FIRST TIME you had sex? _ years old 

37. Think about the FIRST TIME you had sex. Did you or your partner do something or use 

something to stop a pregnancy from happening? 

yes (Go To Question 37a.) 

no (Go To Question 38.) 
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37a. What land of protection did you or your partner use to stop a pregnancy from 
happening the FIRST TIMK you had sex? 
(CHECK ONLY ONE) 

— used a rubber (condom) WITH birth control pills 

_ used a rubber (condom) WITH foam, TODAY sponge, or insert 

— used only birth control pills 
used only a rubber 

— used only the TODAY sponge 
used only foam or insert 

— used a diaphragm (DI-A-FRAM) 

— had sex during the safe time of the month (rhythm) 

— pulled out before sperm came out (withdrawal) 
washed out after sex (douche) 

other 

38. During the LAST FOUR WF^ s, how many 
times did you have sex? times 

39. When was the LA3U0ME you had sex? 

— during the last 4 weeks 2-6 months ago 

— 7-12 months ago over 12 months ago 

I don't remember 

40. Think about the LAST TIME you had sex. Did you or your partner use something to stop a 

pregnancy from happening? 6 F 

yes (Go To Question 40a.) 

no (Go To Question 41.) 

40a. What ^kind of protection did you or your partner use to stop a pregnancy from happening 
che LAST TIME you had sex? (CHECK ONLY ONE) ^ * 

— used a rubber (condom) WITH birth control pills 

— used a rubber (condom) WITH foam, TODAY sponge, 

or insert 

used only birth control pills 

used only a rubber 

used only the TODAY sponge 

used only foam or insert 

used a diaphragm (DI-A-FRAM) 

— had sex during the safe time of the month (rhythm) 

— pulled out before sperm came out (withdrawal) 
washed out after sex (douche) 

other 
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40b. Where did you or your partner get something to stop a pregnancy from happening the 
LAST TIME you had sex? 
(CHFCK ONLY ONE) 

a family planning clinic 

the health department 

a hospital clinic 

a military clinic 

a private doctor 

a drug store 

a friend or relative 

other _ 

I don't know (my partner got the protection) 

— does not apply (pulled out, washed out, or had sex during 
the safe time of the month) 

41. If you have a regular or special piace where you go to get protection, 

where do you usually go? 
(CHECK ONLY ONE.) 

a family planning clinic 

the health department 

a hospital clinic 

a military clinic 

a private doctor 

a drug store 

a friend or relative 

other 

I don't have a special place 

42. Have you EVER used birth control protection when you had sex? 

yes (Go To Question 43.) 

no (Go To Question 44.) 

43. Please CHECK below ALL the reasons why you have used birth control protection. 

— I did not want to get pregnant/ get my girlfriend pregnant 
I did not want to get a disease. 

I did not want to get AIDS. 

My friends use protection and told me to use it 

A friend gave me protection to use. 

My girl/boyfriend wanted us to use protection. 

My mo f Her told me to use protection. 

My fauicr told me to use protection. 

Someone at a clinic told me to use protection. 

— I knew that I was going to have sex and I was prepared. 
Other reason 

44. Have you EVER had sex WITHOUT using birth control protection? 

yes (Go To Question 45.) 

— no (FEMALES: Go To Question 46.) (MALES: Go To Question 49.) 




45 



52 



45. 



Please CHECK below ALL the reasons why you did not use protection. 
T didn't know about birtn control protection, 
didn't care if I got pregnant (my partner pregnant) 
y anted to get pregnant (get my partner pregnant.) 
just didn't think I would get prcgnait (get my partner pregnant), 
thought I was too young (my puitner was too young) to get pregnant 
didn't think I had sex often enough to get pregnant (get my partner pregnanO 
didn't expect to have sex, it was not planned, 
thought it was wrong to use birth control protection, 
thought it was wrong to plan for sex. 
thought birth control was my partner's responsibility. 
My partner didn't want me to use birth controL 
¥ was waiting until I was closer to my boyfriend (my girlfriend.) 
thought my parents had to be told. 

was afraid my family would find out if I used birth control protection, 
thought birth control protection was dangerous to use. 
thought you weren't allowed to get birth control until you were older, 
thought birth control protection cost too much, 
didn't know where to go to get birth control protection, 
t was too hard to get all the way to a clinic to get birth control protection, 
felt uncomfortable going to a strange clinic, 
was afraid to be examined. 

thought birth control protectiori would reduce the pleasure of sex. 
thought birth control protection would be messy to use. 
just didn't get around to it 
_ Other (What? , 



THE FOLLOWING QUESTIONS ARE FOR FEMALES ONLY 
MALES SHOULD GO TO QUESTION 49. 

46. Have you EVER gone to a health clinic to get birth control protection? 
yes no 
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How many times, if ever, have you been oregnant? 

never (Go to Question 51.) 

one time (Go to Question 48.) 

two or more times (Go to Question 48.) 

The LAST time you were pregnant, what did you do? 

I am (NUMBER) months pregnant now. 

I had a baby in (MONTH) 19 . 

I had an abortion in (MONTH) \Z . 

I had a miscarriage in (MONTH) 19 , 
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THE FOLLOWING QUESTIONS ARE FOR MALES ONLY 
FEMALES SHOULD GO TO QUESTION 60. 



49. How many times, if ever, have >ou gotten someone pregnant? 

never (Go to Question 51.) 

one time (Go to Question 50.) 

two or more times (Go to Question 50.) 

I don't know 

50. The LAST time you got someone pregnant, what did she do? 

She is (NUMBER) months pregnant now. 

She had a baby in (MONTH) 19 . 

She had an abortion in (MONTH) 19 

She had a miscarriage in (MONTH) 19 , 

I don't know what she did. 

51. Have you ever had VD (a sexually transmitted disease)? 

yes (Go To Question 51a.) 

no (Go To Question 54.) 

51a. What types? (CHECK ALL THAT APPLY.) 

trick (trichomoniasis) 

clap (gonorrhea) 

NGU (chlamydia) 

yeast infection 

heroes 

crabs 

other 

I don't know the name(s) 

51b. Where did you get treatment? 

a private doctor 

a VD clinic 

the health department 

a hospital clinic 

a military clinic 

other 

— I did not get treatment 

52. Have you had VD (a sexually transmitted disease) during the past 1? months? 

yes (Go To Question 54.) 

no (Go To Question 54.) 

53. If you have NEVER had sex (made love, done it), please CHECK ALL the reasons 
you have not: 

— I think it is wrong to have sex before marriage. 

— My church says it is wrong to have sex before marriage. 
I am not ready to have sex. 

— I am waiting for the right person to do it with. 

— I am waiting until I get married. 
I am waiting until I am older. 

— I do not want to get pregnant/or get someone pregnant. 
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I do not want to get a disease. 

I do not want to get AIDS. 

— My friends think it is wrong to have sex at our age. 

— My parents would be upset if I had sex and they found out. 

— I would be embarrassed to have sex. 

— I do not have a boy/girlfriend to have sex with. 

— I do not know where to get protection to use. 

— I would be embarrassed to use protection. 

— ! do not have enough money to buy protection. 

— I think using protection would make me sick or mess up my body 
Other reason 

54. CHECK ALL of the things below that you have questions or concerns about: 

— how much you weigh your breathing 

— how tall or short you are your stonach 

(pain, vomiting) 

— your skin (rash, acne) the way you look 

— your eyes (vision) your sex organs 

— your ears (hearing) your back 

— your nose your arms 

— your teeth y0U r chest or breasts 

— your legs (cramping) coking 

— W drugs 

— AIDS geiting a job 

— sex (making love, doing iO going to college 

using birth control dying 

— being pregnant sexual abuse 

— getting someone pregnant monthly periods 

— being gay (homosexual) wet dreams 

— sexually playing with yourself dating 

— ra P e yourself 

— the way you think you r friends 

— your ability to leam your grades 

— having nothing to do your parents 

— your brothers/sisters making friends 

— not fitting in with others your teachers 

— family members who drink too much alcohol 
not having a girlfriend or boyfriend 

55. Please CHECK how often you do the things below: 

Never or Once in About once Several Everyday 
or rarely a while a week 



Brush your teeth 
Floss your teeth 
Eat breakfast 
Eat fruits and 
vegetables 
Fat breads, grains, 
and cereals 
Drink milk or eat 
milk products 
Eat meat or fish 



ames 
a week 
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Eat candy or 
snack foods 
Eat Fast Food 
(e.g., McDonald's) 
Lose your appetite 
Drink beer or wine 
Drink hard liquor 
Smoke cigarettes 
Smoke marijuana 
or hash 

Get at least 30 
minutes of good 
exercise (sports, 
jogging, biking) 
Wear seatbelts 
when riding in 
a car 

Drive a car MORE 
THAN 10 miles per 
hour over the 
speed limit 
Drive a car 
wf ,i you have 
been drinking 
Ridfi in a car 
going MORE THAN 
10 miles per 
hour over 
the speed limit 
Ride in a car when 
the driver has 
been drinking 
Feel depressed 
or sad 

Feel angry or mad 
Get in a physical 
fight with someone 
Cany a knife or 
other weapon 
Get at least 7 
hours of sleep 
at night 

Have a hard time 
going to sleep 
at night 



Never or Once in About once Several Everyday 
or rarely a while a week times 

a week 
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56. We would like to hear what you think about your school having a school health center 
Please wnte down any comments, good or bad, you have about the school health center. 



THANKS VERY MUCH FOR FILLING OUT THIS QUESTIONNAIRE. 

NOTE: THE FOLLOWING QUESTIONS SHOULD BE ADDED TO THE VERSION OF THE 
STUDENT HEALTH SURVEY WHICH IS ADMINISTERED TO THE STUDENTS AFTER THE 
CLINIC HAS OPENED. 

Now we would like to ask you a few questions about your school clinic. 

1. Have you ever been to the school clinic for ANY reason? 

yes (Go To Question 2.) 

no (Go To Question 9.) 

2. How many times have you EVER been to 

the school clinic for ANY reason? times 

3. How many times have you been to the school clinic since school 

started this year? times 

4. Which services have you used at the school clinic? 

(CHECK ALL THAT YOU HAVE USED.) 

— counseling (talked to someone about health or personal problems) 
physical exam for school/sports 

first aid for a small injury 

emergency treatment for a big injury 

care for sickness (sore throat, a cough, etc.) 

medicine pick up 

birth control information 

birth control supplies (pills or condoms) 

pregnancy test 

care during or after pregnancy 

STD (VD) tests 

shots (immunizations) 

vision or hearing test 

weight or nutrition education 

counseling for drugs or alcohol 

dental services 

— suggestions for help from other agencies 
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5. How many times, if ever, have you gone to the school clinic to r,et birth control 

protection? times 

6. How satisfied are you with the services you receive jl the school clinic? 

— very satisfied somewhat satisfied 

somewhat unsatisfied very unsatisfied 

7. How comfortable do you feel at the school clinic? 

— very comfortable somewhat comfortable 

— a little uncomfortable very uncomfortable 

8. Do you feel that your visits to the school clinic are secret? 

yes no 

9. Will you keep using the school clinic in the future? 

yes no 

10. Why do you use the school clinic? (CHECK ALL THAT APPLY.) 

I feel it's a part of my school and I can trust it 

It's easy to get to. 

It's the only clinic I know about 

It has the best hours. 

It's the cheapest place I know about 

The people there really care about young people. 

— The people there don't tell my parents I come. 
My friends go there. 

_Other fc 

SKIP THE NEXT QUESTION: GO TO LAST QUESTION. 

11. If you have NOT used the scaool clinic for any reason, why not? 

(CHECK ALL THE ANSWERS THAT APPLY.) 

— My parent(s) did not sign the permission form to use the clinic. 
I was healthy and did not need the clinic. 

I did not need the clinic for birth control 

— I wanted to go on with a clinic I'd been using before. 
I didn't know about the clinic. 

I didn't know where the clinic was. 

I didn't like the staff at the clinic. 

I was not comfortable there. 

I was afraid teachers would find out 

I was afraid my friends would find out 

I was afraid my parents would find out 

My friends told me the clinic was not any good. 

I thought the clinic cost too much money. 

The clinic is too close to school. 

The clinic is too far from where I live. 

— The clinic did not have the kind of health care that I wanted. 
I just didn't get around to it 

I didn't want to miss class. 

Other (What? 
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12. Do you think you will use the school clinic in the future? 
— yes no 

lLd e Uke°!o^ell ke us i0 kn ° W y ° U 3b0Ut SCh ° 01 ° liniC - PIeaSC Wrfte d0wn y° u 



THANKS VERY MUCH FOR FILLING OUT THIS QUESTIONNAIRE. 
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Appendix B: The CPO School-Based Clinic 
Management Information System 



Introduction 

This document describes the management information system developed by the Center for 
Population Options (CPO) for school-based clinics across the country. CPO developed this system 
with three goals in mind: (1) to help clinic sites gather information that will facilitate good clinic 
management and patient care, (2) to provide a uniform data collection system so that SBCs across 
the country can communicate with each other and with potential funders in a straightforward, 
consistent way, and (3) to build a national database of summary statistics from all school-based 



Given the controversy surrounding the school-based clinic movement, increasingly it will 
become important for SBCs to document what they do and how well they work in order to attract 
and maintain financial and community support. Some clinic sites have contacted CPO for 
information on how to develop and implement management information systems. Other sites need 
technical assistance to improve the management of their clinics and the preparation of statistical 
reports on service delivery. Consequently, CPO has developed a standardized management 
information system that includes both data collection forms and IBM-compatible microcomputer 
software for processing and analyzing encounter data. 



Important Features of the Management Information System 
In general, the standardized management information system: 

o collects background information separately from encounter information so that only 
subsequent new information is collected each encounter (e.g., for each encounter the 
patient's demographic data does NOT have to be collected); 

o conforms with federal BCRR itandards 1 for reporting clinic encounter data and uses 
ICD-9-CM codes for recording diagnoses; 

o requires that only three data collection forms be used; 



permits the introduction of one or more optional forms over time as needs change 
(there are six such supplemental forms); 



BCRR definiuons for encounters and users (taken from the Instruction Manual for the BCHS Common 
rang Requirements, revised January 1982): (a) ENCOUNTERS - A focfcfifcfece. contact between a user 
1 Pr °? vf . 0X6 services wh0 exercises independent judgment in the provision of health services 
ie individual pauent. For a health service to be defined as an encounter, the provision of the health 



climes. 
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is currently written in dBase m-Plus, a popular software language that can be 
revised by sites if needed; 



is a menu-driven system that is easy for people without microcomputer experience 
to learn and use; 

can generate approximately 100 pre-programmed statistical tables on clinic services 
and user characteristics (nearly one-half of these tables can be produced using only 
the three required forais); 

is designed for IBM-compatible microcomputers with 640K and a 20 megabyte haid 
disk and an Epson compatible printer, and 

can be tailored to each site, when appropriate, to reflect the particular needs of that 
site while maintaining a standardized format. 



Benefits to Sites in Using This System 

School-based clinics that utilize CPO's management information system will realize a wide 
variety of tangible benefits: 

o Use of the system will facilitate good patient care and provide the information 

necessary to improve clinic efficiency. The process of form completion encourages 
clinic staff to consistendy probe patients for important health-related information that 
should promote the delivery of comprehensive services. 

o The system is capable of producing pre-programmed tables that will help clinic 
managers allocate staff and inancial resources in needed areas to improve clinic 
services and patient outcomes. Such tables include statistics on: 

the major purposes of visits, 
diagnoses made, 

types of medical procedures and counseling services provided, 

types of external referrals made, 

staff utilization patterns, and 

a wide variety of other useful stannary data. 

o The statistical reports can also be used to document the success of the clinic in 
delivering a range of services to eligible students, thus building a strong case for 
new or continued funding. 



The required hard disk capacity will depend on the expected number of encounters per year at a given 



site. 
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Management Infomiatinn System Services and Materials 



The Management Information System has two major parts: the clinic visit forms and a 
specially written software program for data entry, analysis and report generation 5 . Sites that choose 
to use this system will receive: 

o samples of three required data collection forms 4 , 

o samples of six optional forms that collect supplemental information on a wide range 
of clinic activities and student health (i.e., lab test and referral activity logs, specific 
lab test results, appointment compliance for onsite visits and for external referrals, 
prenatal care outcomes, health and psychosocial history, behavioral risks to health 
status), 

o an instruction manual for form completion, 

o a disk copy of the management information system software program, 

o an instruction manual for the software program, and 

o an annual report describing the national database findings. 

Management and Research Questions that Can Be Answered With the CPO 
Management Information System 

Questions That Can Be Answered With the Three Required F n rms fThe Reg is tration Form the 
Standard Visit Form, an d the Case Closing Form 

Identification of Specific Clinic Users 

1. What are the names, clinic ID numbers, sex and birth dates of all the students 

registered in the clinic? What was the date of the last visit for users and what was 
the major purpose of that visit? 



Some sites have a relatively small number of encounters per year (less than 500) and do not have 
access to a microcomputer. These sites can request the three basic data collection forms and a manual 
describing how to complete the forms. In addition, such sites will receive instructions on how to calculate 
important summary statistics contained in several tables for their own internal use as well as for the CPO 
National Database Project 



These three forms include a REGISTRATION FORM that records basic demographic information on 
each pauent; a STANDARD VISIT FORM that documents the purpose of the visit, diagnoses made, medical 
procedures and counseling provided, and referrals made; and a CASE CLOSING ACTIVITY LOG that 
provides a means to isolate active from inactive cases for statistical reporting purposes, and records whv 
specific cases are closed. 3 
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Summary Statistics on Clinical Utilization and Diagnoses 

2. How many students are registered for care in the school-based clinic each reporting 
penod (i.e., have a completed REGISTRATION FORM)? 

3. How many students utilize the school-based clinic each reputing period (i.e., have 
at least 1 encounter with a provider)? How many users are new users? What is 
the breakdown of clinic users (new and return users) by grade level, age. sex. and 
ethnicity? 

4. How frequently do students use the school-based clinic? Do different students use 
the clinic more frequently (i.e., what are the breakdowns by grade level, age sex 
and ethnicity)? * 

5. How many visits are there for each purpose of visit (as defined by the student and 
the medical provider)? How do these purposes differ according to grade level, age 
sex, and ethnicity? What are the purposes of students' fira visit? 

6. What types of counseling and health procedures are provided? How do the numbers 
of counseling encounters and health procedures differ by grade level, age, sex, and 
ethnicity? 

7. Which specific laboratory tests are performed in the school-based clinic? How 
many of each type are performed? 

8. What health and psychosocial problems are diagnosed? How many users are 
diagnosed with each health and psychological problem? What are the number of 
visits for each type of diagnosis? What is the breakdown of users with different 
diagnoses by grade level, age, sex, and ethnicity? (Diagnoses will be based on 
ICD-9-CM codes.) 

9. What types of referrals are made to other agencies? 

10. What is the number and percentage of visits that are scheduled vs. walk-ins? 

11. When are students seen in the school-based clinic? What is the breakdown by 
month, day of week, and time of day? 

12. On the average, how many encounters do different types of staff members have 
each sporting period? What is the average number of encounters per visit? 

13. How many cases are closed each reporting period? What are the reasons for closing 
cases? How many closed cases are referred elsewhere for care? 
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Summary Statistics on Student Health Characteristics and Outcomes 3 



14. Weight control: Among students attending the clinic, what is the prevalence of 
obesity? What is the prevalence of underweight students? What is the number and 
percentage of under- and over-weight students visiting the clinic for weight 
management? What percent of underweight students are diagnosed as anorexic? 
What percent of overweight students are diagnosed as obese? What weight change 
occurs over time in students participating in weight management programs? 6 

15. Blood pressure: Among students using the clinic, what is the number and percent 
with abnormal blood pressure? 

16. STD Prevention: Among students attending the clinic, what is the prevalence of 
STDs? What is the breakdown by grade level, age, sex, and ethnicity? 

17. Pregnancy Prevention: What percent of clinic users have received contraceptive 
prescriptions and/or supplies in the clinic? What percent are referred to other sites 
for pregnancy prevention services? What percent of female clinic users are 
identified as pregnant by ICD-9-CM code? What is the breakdown of pregnant 
students by grade level, age, and ethnicity? What percent of pregnant students have 
a repeat pregnancy? 

18. Mental Health: What percent of students are diagnosed as depressed or having 
adjustment problems? What percent of students receive counseling or therapy at the 
clinic or are referred for counseling or therapy? 

Additional Questions That Can Be Answered with the Pers o nal Health Survey - Part c. an Op ting 
Form 



Identification of Clinic Users and Specific Students for Intervention and/or Follow-Up 

1. Which students report symptoms of depression or adjustment problems? Which 
students report they have engaged in certain risk-taking behaviors, for example, 
unprotected intercourse, smoking, drinking, or using drugs? 

Summary Statistics on Student Health Characteristics and Outcomes (see footnote #5) 

2. Prior medical care use: Among students using the clinic, how much time has 
elapsed since they received medical and dental care prior to survey administration? 

3. Absenteeism: Among students using the clinic, how many days of school did they 
miss due to illness during the semester prior to survey administration? 



Whether or not the questions listed in this subsection can and should be answered depends upon the 
services offered by the clinic. Some of the questions are not appropriate for some clinics. Other clinics may 
not wish to collect systematically the information needed to answer these questions. 

This visit form/software system will provide some of the raw data needed to answer this question, but 
addiuonal work will need to be done by hand or via special software programming. It is likely that someone 
with a research background will be needed to conduct these special analyses. 
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Immunization Compliance: At survey administration, what percent of students have 
complete immunizations according to the state and AAP standards for school 
attendance? 

Pregnancy Prevention: At survey administration, what percent of students have ever 
had sexual intercourse? Among those who have never had sex, what are their 
reasons for delaying intercourse? What percent of clinic users used a contraceptive 
method the last time they had intercourse? Among those using a method at last 
intercourse, what method did they use? If no method was used at last intercourse, 
what were the reasons for non-use? How many students report they have had an 
STD, and what types of STDs have they had? How many females and males report 
they have been pregnant (or gotten someone pregnant) and what were the outcomes 
of those pregnancies? Overall, what is the parenting status of students? 

Substance Use: Among students attending the clinic, what percent provide 
indications that they are smoking, using drugs or drinking excessively? What is the 
breakdown by grade level, age, sex, and ethnicity? 

Mental Health: Among students attending the clinic, what percentage report they 
are frequently nervous or depressed, or report other signs of mental distress? 



Additional Onestinps That. r> Anyn ed with the Visit Form Up da te, an Optional Form 
Identification of Clinic Users and Specific Students for Intervention and/or Follow-Up 

1. Which students are not completely immunized? (Note : In order to address this 
question, both the Visit Fonn Update and the Personal Health Survey - Part C must 
be used.) 

Summary Statistics on Diagnoses and Follow-Up 

2. What percentage of the results for specific types of lab tests are normal versus 
abnormal or positive versus negative? 

3. What percentage of first appointments to external referral agencies are kept? What 
percentage of missed first appointments are rescheduled? What percentage of 
rescheduled first appointments are kept? 

4. Among students referred for contraceptive supplies, what proportion get those 
supplies? 

Summary Statistics on Student Health Characteristics and Outcomes (see footnote #1) 

5. Immunization Compliance: Among students who were qoj compleielv immunized at 
administration of the Personal Health Survey - Part C, what percent have complete 
immunizations according to the state standards for school attendance as a result of 
clinic intervention? (Note: In order to address this question, both the Visit Form 
Update and the Personal Health Survey - Part C must be used.) 
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Addit i ona l Questions That Can Be A n s wered with the naiiv Activity r.n ? . an optional Earn 



Summary Statistics on Appointment Compliance 

1. What is the rate of compliance for scheduled appointments? What arc the 
compliance rates for different types of appointments (e.g., STD care, prenatal care 
drug abuse, family problems)? 

2. How are appointment compliance rates affected by grade level, age, sex and 
ethnicity? 



Additional Questions That Can Be Answered with the Prenyl C are and JMfgg Oata Form, an 
Optional Form 

1. Among students attending the clinic, what percentage of pregnant students receive 
prenatal care onsite? What percentage are referred offsite? How many previous 
pregnancies have new mothers had? How many months elapse between conception 
and preg»^ncy diagnosis? How many weeks elapse before prenatal care is initiated? 
How many prenatal visits are completed before birth? What percent of infants have 
a low birth weight and/or are premature? What percent of mothers and infants have 
prenatal and intrapartum complications? What percent of student mothers return to 
school after birth? What is the breakdown of mothers who return and do not return 
to school by primary caretaker status? 



For more information about the CPO Management Information System or the National 
Database Project, please contact: 



Cindy Waszak 
Center for Population Options 
1012 14th St. N.W., Suite 1200 
Washington, D.C., 20005 
(202) 347-5700 
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Appendix C: Methods for Assessing A Clinic's 
Pregnancy Prevention Program 

Background 

To assess the views of clinic practitioners on important characteristics we completed several 
tasks. First, we reviewed the family planning literature, talked with family planning professionals 
and researchers in the field, met with SEC clinic staff, and developed an initial list of potentially 
important characteristics of SriC pregnancy prevention programs. 

Second, several SBC practitioners reviewed the list and suggested additions and changes. 
Most of these were incorporated, producing a list of 108 pote*. ally important characteristics. 

Third, we asked 28 SBC practitioners to rate these characteristics. Twenty-four of them did 
so. They included professionals working in clinics in different pans of the country and with 
different reproductive health policies and procedures (e.g., some that refer, some that prescribe; and 
some that dispense). They included staff from both larger and smaller clinics. Finally, they 
included clinic educators, nurse-practitioners, and administrators. 

Despite the variety of clinics, clinic policies, and positions within clinics from which this 
sample was drawn, they all had a wealth of experience in clinics. Most of them have worked with 
teens on a daily basis; have seen teens' reactions to clinic staff and procedures; have observed 
changes in clinic utilization when clinic programs or staff changed; and have had discussions with 
teens about many different things. In general, this panel of 24 professionals represents an important 
body of experience and insight into school-based clinic operations. 

The panel rated each potentially important characteristic on a 1-10 scale, from not at all 
important to extremely important All items with a mean rating greater than 7.0 are included in the 
list below. 

Directions for Assessing the Clinic 

This list can best be used by clinic staff or a site visit team io assess whether or not the 
clinic has each of the characteristics rated important That is, it can best be used qualitatively. 

It is also possible to use this list to produce a numerical score for a clinic. The total 
numerical score can provide a rough indication of the overall quality of the reproductive, health 
program. However, these numerical scores should be used cautiously, because not all of the 
characteristics in the list have equal weight 

To produce a numerical score: 

1. Use the following scale to rate the clinic on each characteristic on the list: 

0 = The clinic does not have this characteristic. 

1 = This clinic has this characteristic to some extent, but it is not well 

implemented. 

2 = The clinic has this characteristics and it is well 

implemented. 

2. Add all the scores. The maximum possible score is 190. 
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Reproductive Health and Pregnancy Prevention Inventory Programs 

ACCESS 

The clinic is open at least 20 hours during each week, 

— Teachers allow students to attend the clinic during school nours without hassle. 

— Students can go to the clinic during their free periods. 

— Students can go to the clinic before or after school. 

— If the clinic is not open during the summer months or other vacation periods, arrangements 
are made for alternative providers during these periods. 

— The clinic is open during the summer months and other vacation periods. 

— If parental consent is required for services, the clinic has an effective method in place for 
obtaining parental consent 

— The clinic obtains blanket parental consent (as opposed to itemized consent) for all services, 
including family planning. 

MEDICAL SERVICES 

Condoms are provided at the clinic. 

— Birth control pills (or other medical methods) are prescribed at the clinic, 

— Birth control pills (or other medical methods) are dispensed at the clinic. 
Pregnancy tests are available. 

— Pregnancy test results, both positive and negative, are given only in person. 
COUNSELING SERVICES 

— Counseling is made available to all students on decisions about having sex and using birth 
control during routine visits as well as during reproductive health visits. 

— Abstinence and the decision to be sexually active are discussed with all new family 
planning clients. 

— Teens are strongly encouraged not to have sex without birth control unless they want to 
have a baby. 

— Minor side effects o* pills (e.g., weight gain, nausea, breakthrough bleeding) are discussed 
with clients using or considering the use of oral contraceptives. 
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— Clinic staff discuss decisions about sex in a way that does not alienate students. 

— The clinic counsels students on legal pregnancy options, or refers students to an 
appropriate agency for pregnancy counseling. 

— Staff encourage students to discuss their decisions about sexuality with their parents. 

— There is counseling available for males alone. 

— There is counseling available for teen couples. 

— Parent and parent-teen counseling sessions are available. 

— Counseling is available without medical services. 

— Counselor recognize the different developmental phases of adolescence and counsel 
accordingly. 

APPOINTMENTS 

— Students do noi have to wait more than one week for a family planning appointment 

_ Students can walk in and normally be seen by someone (:iot necessarily a clinician) instead 
of having to wait up to a week. 

_ Students can normally walk in and be seen for a new contraceptive visit (instead of having 
to wait up to a week). 

— Students can walk in and be seen for a continuing contraceptive visit (instead of having to 
wait up to a week). 6 

— Students can walk in and be seen for a pregnancy test (instead of having to wait up to a 
week). e 

— Students are given at least 45 minutes for their first family planning visit. 
PROTOCOLS 

_ There are established protocols for all medical procedures that have been approved bv a 
physician. ' 

Whenever staff conduct a general health assessment on any client, questions about sexuai 
activity and use of birth control are asked and follow-up services are provided as 
appropriate. 

Whenever reproductive health services are provided, a general health assessment is obtained, 
including the adolescent's own health concerns. 

_ An assessment of general behavioral risk-taking is done as a part of reproductive health 
services. 

— Contraceptive compliance k checked at every visit regardless of the purpose of the visit 
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— If a student is having sex but not using birth control, that student is promptly given an 
appointment for counseling and/or contraceptive care, as appropriate. 

— Frequency of famiiy planning visits is determined on the oasis of individual need. At a 
minimum, monthly follow-up is done until the client is using his/her selected method 
appropriately. 

_ If a patient is having problems with a particular family planning method, another method is 
dispensed as appropriate. 

— A reasonable protocol is in place for students who use condoms to come back for continued 
counseling and supply pick-up or referral. 

— Case management and conferencing is done for all family planning clients who appear to be 
inconsistent users. 

— The clinic has a system in place for reminding clients of upcoming appointments. This 
system maintains the confidentiality of the purpose of the visit. 

— When a student misses a family planning appointment, the clinic has a tickler system for 
recontacting the student. 

— If piUs or other methods of birth control are not dispensed, the clinic refers student to a 
particular provider that does dispense. 

— A physical examination is not required for nonprescription methods. 

— For those students who are getting nonprescription methods and who have not had a 
physical examination, getting a physical examination is encouraged. 

CONFIDENTIALITY 

— Students understand that their parents will not be notified about visits for family planning 
without the students' consent. 

— Going to the clinic does not indicate that the purpose of the visit is for family planning 
(e.g., students know that many students go to the clinic for other purposes and students do 
not know that visits during particular hours means that the purpose is for family planning). 

— Other students do not work in the clinic and see students' recoitis. 

— Students know that their clinic records will not be shared with the school authorities. 

— The clinic uses a variety of ways to communicate to students that family planning services 
are confidential (e.g., signs/posters in the waiting room, confidentiality is discussed in the 
counseling room, or a statement about confidentiality is included on the intake form). 

OUTREACH 

— Staff give presentations in .school classrooms to enhance visibility and acceptance on 
campus. 
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— In classroom presentations, staff describe the full range of clinic services, including the 
reproductive health services. 

— Staff put up posters, give infonnauon to students during school registration, have columns 
in the school newspaper, or use some other method of adequately informing the students 
about the clinic's family planning services. 

Clinic staff coordinate with school staff, especially the school nurse, teachers, and 

administrative staff, to facilitate referral and follow-up. 

SEX EDUCATION IN THE CLASSROOM 

— Within the school, there is a sound sex education program that covers the major topics in 
sexuality: decisions about abstinence and having sex, birth control, the probability of 
pregnancy, and STD. 

— Within the school, there is a sex education program that provides considerable role playing 
and practice in saying no to sexual activity and insisting on the use of birth control when 
planning to have sex. 

Clinic staff provide or assist with the sex education programs in the classroom. 

GROUP SESSIONS 

There are group rap sessions on sexuality available to students during or after school 

either in the clinic or linked with the clinic. 

The sizes of these sessions are kept small so that students have ample opportunity to ask 

questions and to discuss their feelings about sexuality. 

These are sufficiently well organized and advertised so that over time many students 

participate in them. 

STAFF 

Female clinicians and counselors are available. 

Male clinicians and counselors are available. 

Staff have genuine warmth, empathy, openness, concern, and respect for teens. They like 

to work with teens and have excellent rapport with teens. 

Staff have previous experience working with teens. 

Staff receive special training in adolescent development and medicine appropriate for their 

position. 

Clinicians and counselors are given periodic in-service training. 

Staff have trailing in sexuality. 

All clinicians have training in pelvic assessment. 
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WAITING ROOM 

— The waiting room is attractive and appealing to teenagers. 

— The waiting room has pamphlets on sexual decision-making, contraception, pregnancy, 
STDs, and other family planning themes. 

— The waiting room has posters on sexual decision-making, contraception, pregnancy, STDs, 
or other family planning themes. 

COUNSELING AND EXAMINATION ROOMS 

— Counseling and examination rooms are reasonably comfortable and private, allowing for 
confidential services. 

MATERIALS 

— There are pamphlets on the decision to have or abstain from sex. 
There are pamphlets on contraception. 

There are pamphlets on STDs and AIDS. 

— There are pamphlets of each type appropriate for both males and females. 

— There are culturally-sensitive pamphlets on all topics, as appropriate. 

— The clinic has a variety of visual aides for counseling and education such as male and 
female pelvic models, samples of all family planning methods, and hand mirrors for 
observing the genitals during pelvic examinations. 

COSTS 

— Services at the clinic are free or sufficiently low so that cost is not a barrier to access for 
c tudents. 

— Pills, condoms, and other methods of birth control are free or cost only a small amount. 

— If the clinic charges for specific services (e.g., certain lab tests) or has an annual users' 
fee, partial payment is allowed when students cannot pay the full charge. 

— Sliding fee scales are used in clinics that charge for services. 

_ Referral agencies provide free or low-cost care to students, including prescription drugs 
and supplies. 

COMMUNITY RELATIONS 

_ Clinic tours are available to students, parents, school staff and community members upon 
request 
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— Staff give presentations to parent groups and other community groups. 

— The ciinic has a student advisory board, or some other procedure for gering input from 
the students on how to improve the clinic. 

— The clinic has a good reputation among the students. 

— The clinic is well accepted and supported by the school teachers and administration. 

— The clinic has an adult advisory board, or some other procedure for getdng input from 
parents and adults in the community on how to improve the clinic. 

— The clinic has the support of the parents and community more generally. 

— The provision of quality health care by the clinic is not unduly limited by school, 
community, or state regulations. 

EVALUATION AND ASSESSMENT 

— The clinic has a method established for determining how many teens are getting pregnant 
in the school each year and how many of the clinic's patients are gettjig pregnant 

— The clinic does enough follow-up on pregnant teens to know the major reasons why the 
pregnant teens got pregnant. 

— The results of these assessments are used to improve the program. 
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